Jul 09 18:02:08.014055 ip-10-0-132-102 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 18:02:08.014069 ip-10-0-132-102 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 18:02:08.014079 ip-10-0-132-102 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 18:02:08.014377 ip-10-0-132-102 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 18:02:18.216289 ip-10-0-132-102 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 18:02:18.216308 ip-10-0-132-102 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 88cb204ee85c4f41a31599009845601d --
Jul 09 18:03:56.385317 ip-10-0-132-102 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.845054    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848917    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848931    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848936    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848940    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848943    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848945    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848948    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:56.924668 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848950    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848953    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848956    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848959    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848962    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848964    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848967    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848969    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848972    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848975    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848981    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848984    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848987    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848990    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848992    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848995    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.848997    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849000    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849002    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849004    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:56.928134 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849007    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:56.928079 ip-10-0-132-102 systemd[1]: Started Kubernetes Kubelet.
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849010    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849013    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849016    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849018    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849021    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849023    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849026    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849043    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849046    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849048    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849051    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849054    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849057    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849060    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849062    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849065    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849067    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849070    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849072    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:56.933329 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849077    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849079    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849082    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849086    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849088    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849091    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849094    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849097    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849099    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849101    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849104    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849106    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849109    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849112    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849114    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849117    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849119    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849122    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849125    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849127    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:56.937750 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849130    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849132    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849135    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849139    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849143    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849145    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849148    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849151    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849155    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849158    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849161    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849164    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849167    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849169    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849172    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849175    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849177    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849180    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849182    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849185    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:56.943115 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849187    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849190    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849192    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849194    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849198    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849201    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849204    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849206    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849209    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849211    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849214    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.849216    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851611    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851623    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851628    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851631    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851634    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851637    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851642    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:56.946447 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851647    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851650    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851652    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851655    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851658    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851661    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851663    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851666    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851668    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851671    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851674    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851677    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851680    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851683    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851686    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851688    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851691    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851693    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851696    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851699    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:56.951554 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851703    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851707    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851711    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851715    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851718    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851722    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851725    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851728    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851731    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851734    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851737    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851739    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851742    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851744    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851747    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851749    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851752    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851754    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851758    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851760    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:56.954948 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851763    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851765    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851768    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851776    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851779    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851782    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851784    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851787    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851790    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851792    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851795    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851797    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851800    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851803    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851807    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851809    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851812    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851815    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851818    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851820    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:56.961334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851827    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851830    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851833    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851835    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851838    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851840    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851843    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851845    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851848    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851850    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851853    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851856    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851859    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851861    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851864    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851866    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851869    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851873    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851876    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:56.965149 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851881    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851883    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851886    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851888    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851891    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851894    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851897    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851900    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851902    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851905    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851907    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851910    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.851913    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852334    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852339    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852342    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852345    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852348    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852351    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:56.970942 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852353    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852356    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852359    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852361    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852364    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852367    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852369    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852372    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852375    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852377    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852385    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852388    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852391    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852395    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852398    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852401    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852404    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852406    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852408    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852411    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:57.125991 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852414    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852416    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852419    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852422    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852424    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852427    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852429    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852432    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852435    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852437    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852440    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852444    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852447    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852450    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852453    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852456    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852459    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852462    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852464    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:57.140548 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852467    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852469    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852472    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852475    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852483    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852487    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852492    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852495    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852497    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852500    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852502    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852505    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852508    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852510    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852513    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852515    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852518    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852520    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852523    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852525    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:57.142263 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852529    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852532    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852536    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852538    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852541    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852543    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852546    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852548    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852551    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852553    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852556    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852559    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852561    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852564    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852568    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852571    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852574    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852580    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852584    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:57.143920 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852587    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852590    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852592    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852595    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852597    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852600    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852602    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852605    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852607    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852610    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852613    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852615    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852617    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852620    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.852622    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852693    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852700    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852709    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852715    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852721    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852725    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 18:03:57.146558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852731    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852737    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852740    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852743    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852747    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852750    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852753    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852756    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852759    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852762    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852767    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852770    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852774    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852777    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852781    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852784    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852787    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852791    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852794    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852797    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852800    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852803    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852807    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852810    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 18:03:57.148703 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852813    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852816    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852820    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852823    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852826    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852829    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852832    2567 flags.go:64] FLAG: --enable-server="true"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852835    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852840    2567 flags.go:64] FLAG: --event-burst="100"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852844    2567 flags.go:64] FLAG: --event-qps="50"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852847    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852850    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852853    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852856    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852859    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852862    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852865    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852867    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852872    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852875    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852878    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852883    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852886    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852888    2567 flags.go:64] FLAG: --feature-gates=""
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852892    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 18:03:57.151327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852895    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852898    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852901    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852905    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852908    2567 flags.go:64] FLAG: --help="false"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852910    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852913    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852916    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852919    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852922    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852926    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852929    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852931    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852934    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852937    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852940    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852942    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852945    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852948    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852951    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852954    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852956    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852959    2567 flags.go:64] FLAG: --lock-file=""
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852962    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 18:03:57.153934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852965    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852970    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852975    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852978    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852984    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852987    2567 flags.go:64] FLAG: --logging-format="text"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852990    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852994    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852996    2567 flags.go:64] FLAG: --manifest-url=""
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.852999    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853006    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853009    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853013    2567 flags.go:64] FLAG: --max-pods="110"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853016    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853019    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853022    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853025    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853042    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853046    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853049    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853056    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853059    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853063    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 18:03:57.159376 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853066    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853068    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853074    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853077    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853080    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853083    2567 flags.go:64] FLAG: --port="10250"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853086    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853089    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-049ad5829b8531d79"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853092    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853095    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853103    2567 flags.go:64] FLAG: --register-node="true"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853106    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853109    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853113    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853117    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853120    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853124    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853127    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853130    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853133    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853136    2567 flags.go:64] FLAG: --runonce="false"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853139    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853142    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853145    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853148    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 18:03:57.160193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853151    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853154    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853157    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853160    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853163    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853166    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853169    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853172    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853175    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853178    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853180    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853185    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853188    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853191    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853195    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853197    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853200    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853204    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853207    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853210    2567 flags.go:64] FLAG: --v="2"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853213    2567 flags.go:64] FLAG: --version="false"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853219    2567 flags.go:64] FLAG: --vmodule=""
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853222    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853226    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853347    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:57.167275 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853351    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853354    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853357    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853359    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853362    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853366    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853370    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853373    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853376    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853378    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853381    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853383    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853386    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853388    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853391    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853394    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853396    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853399    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853401    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853404    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:57.168117 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853406    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853408    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853411    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853413    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853417    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853420    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853422    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853425    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853429    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853432    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853435    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853438    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853441    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853443    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853446    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853448    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853452    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853455    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853458    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853461    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:57.168666 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853464    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853467    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853470    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853473    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853475    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853477    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853480    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853483    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853486    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853488    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853491    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853493    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853496    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853498    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853501    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853503    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853507    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853509    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853512    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853514    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:57.169291 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853519    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853522    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853525    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853528    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853530    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853533    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853535    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853538    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853541    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853543    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853546    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853548    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853551    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853554    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853556    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853559    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853562    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853564    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853567    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853569    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:57.169874 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853572    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853574    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853577    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853579    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853582    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853585    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853589    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853592    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853596    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853599    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853601    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853604    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853607    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853610    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853613    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853616    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853619    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:57.170524 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.853621    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.853627    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.860841    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.860858    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.860872    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.860880    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860937    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860944    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860948    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860952    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860956    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860960    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860963    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860966    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:57.171046 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860969    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860971    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860974    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860977    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860979    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860982    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860985    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860987    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860990    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860993    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.860997    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861000    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861007    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861010    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861014    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861017    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861021    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861024    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861042    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:57.171646 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861047    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861050    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861053    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861056    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861059    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861061    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861064    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861067    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861069    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861072    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861074    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861077    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861079    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861082    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861084    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861087    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861091    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861094    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861097    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861099    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:57.276503 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861102    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861104    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861107    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861110    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861112    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861116    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861119    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861122    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861124    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861127    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861130    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861133    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861136    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861139    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861142    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861144    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861147    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861149    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861152    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861154    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:57.277564 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861157    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861160    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861162    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861165    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861167    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861169    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861172    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861175    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861179    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861181    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861184    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861187    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861189    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861192    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861194    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861196    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861199    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861204    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861207    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861209    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:57.278395 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861212    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861215    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861217    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861220    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861222    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861225    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861228    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861230    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861233    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861235    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861238    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861240    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.861246    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861363    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:57.279628 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861367    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861370    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861374    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861377    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861380    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861382    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861387    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861389    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861392    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861395    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861397    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861400    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861402    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861405    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861408    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861410    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861415    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861418    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861420    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861423    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:57.280044 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861426    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861428    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861431    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861434    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861436    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861439    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861441    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861444    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861446    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861449    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861451    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861454    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861456    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861459    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861462    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861464    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861467    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861470    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861474    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861477    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:57.280843 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861480    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861482    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861485    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861487    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861490    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861492    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861495    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861497    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861501    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861504    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861507    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861509    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861512    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861514    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861517    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861520    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861522    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861525    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861529    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861533    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:57.281633 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861536    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861539    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861542    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861546    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861549    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861552    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861555    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861557    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861560    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861562    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861566    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861569    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861571    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861574    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861576    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861579    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861581    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861583    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861586    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:57.282477 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861588    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861592    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861595    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861598    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861600    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861603    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861606    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861608    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861611    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861614    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861617    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861620    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861622    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861625    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861628    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861630    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861632    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861635    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:57.283337 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:03:56.861637    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.861642    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.861748    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.864175    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.865726    2567 server.go:1019] "Starting client certificate rotation"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.865837    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.865879    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.868691    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.870251    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.889939    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.893526    2567 log.go:25] "Validated CRI v1 image API"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.894352    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 18:03:57.284130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.896838    2567 fs.go:135] Filesystem UUIDs: map[4a674ca9-a694-4991-8343-5f694698adac:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 903c6e58-45a9-4cdb-ae9a-2b76981257a9:/dev/nvme0n1p4]
Jul 09 18:03:57.284520 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.896862    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 18:03:57.284520 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.900126    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.904895    2567 manager.go:216] Machine: {Timestamp:2026-07-09 18:03:56.903429131 +0000 UTC m=+0.389090474 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101251 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb4066a9f11a083d11d629de6a3db SystemUUID:ec2eb406-6a9f-11a0-83d1-1d629de6a3db BootID:88cb204e-e85c-4f41-a315-99009845601d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:50:b4:7a:b2:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:50:b4:7a:b2:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:24:e6:0e:98:4c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.904997    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.905102    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.909005    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.909044    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.909848    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.909858    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.909866    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.909882    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.911578    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.914904    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915018    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915055    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915065    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915077    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915086    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915094    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 18:03:57.290214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915412    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.915427    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.917833    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920560    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920588    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920751    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920763    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920769    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920775    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920782    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.920788    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.921363    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.921369    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.921376    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.921382    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.921593    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.921601    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.922556    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.922563    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.927313    2567 server.go:1268] "Started kubelet"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.927643    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.927708    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.927938    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.929102    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.931176    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.936183    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.937938    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.937965    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.937984    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.938258    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.938281    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.938337    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.938955    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.938960    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.938977    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.939078    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.939087    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.937932    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-102.ec2.internal.18c0b127cc3fb96a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-102.ec2.internal,UID:ip-10-0-132-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-102.ec2.internal,},FirstTimestamp:2026-07-09 18:03:56.92728561 +0000 UTC m=+0.412946953,LastTimestamp:2026-07-09 18:03:56.92728561 +0000 UTC m=+0.412946953,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-102.ec2.internal,}"
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.943687    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.943702    2567 factory.go:55] Registering systemd factory
Jul 09 18:03:57.291055 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.943716    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.943944    2567 factory.go:153] Registering CRI-O factory
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.943960    2567 factory.go:223] Registration of the crio container factory successfully
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.943985    2567 factory.go:103] Registering Raw factory
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.944001    2567 manager.go:1196] Started watching for new ooms in manager
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.946936    2567 manager.go:349] Starting recovery of all containers
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.951726    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.951742    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.962559    2567 manager.go:354] Recovery completed
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:56.963963    2567 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.967600    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4xnxc"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.967708    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972119    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972147    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972157    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972763    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972777    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972777    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.972819    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.973613    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.973637    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.973647    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.974164    2567 policy_none.go:49] "None policy: Start"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.974180    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.974193    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.975227    2567 policy_none.go:47] "Start"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:56.976350    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4xnxc"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.022792    2567 manager.go:337] "Starting Device Plugin manager"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.022833    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.022847    2567 server.go:87] "Starting device plugin registration server"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.023194    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.023214    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.023317    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.023404    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.023437    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.023920    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.023976    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.048129    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.049342    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.049377    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.049403    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.049437    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.052040    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.124292    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.125204    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.125227    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.400410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.125237    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.125259    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.138429    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.138453    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-102.ec2.internal\": node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.150337    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"]
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.150609    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.151886    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.151921    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.151933    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.168889    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.168946    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.168980    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.169796    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.169801    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.169823    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.169827    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.169836    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.169842    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.171066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.171090    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.171827    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.171851    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.171860    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.240836    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c39ad98931262a830867440cbd12b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal\" (UID: \"03c39ad98931262a830867440cbd12b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.240878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c39ad98931262a830867440cbd12b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal\" (UID: \"03c39ad98931262a830867440cbd12b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.240907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33d903d9ca629bd1f13fc91f98e8c89a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-102.ec2.internal\" (UID: \"33d903d9ca629bd1f13fc91f98e8c89a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.289774    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.341640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c39ad98931262a830867440cbd12b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal\" (UID: \"03c39ad98931262a830867440cbd12b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.341665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c39ad98931262a830867440cbd12b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal\" (UID: \"03c39ad98931262a830867440cbd12b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.401992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.341687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33d903d9ca629bd1f13fc91f98e8c89a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-102.ec2.internal\" (UID: \"33d903d9ca629bd1f13fc91f98e8c89a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.403177 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.341721    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33d903d9ca629bd1f13fc91f98e8c89a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-102.ec2.internal\" (UID: \"33d903d9ca629bd1f13fc91f98e8c89a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.403177 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.341748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c39ad98931262a830867440cbd12b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal\" (UID: \"03c39ad98931262a830867440cbd12b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.403177 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.341721    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c39ad98931262a830867440cbd12b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal\" (UID: \"03c39ad98931262a830867440cbd12b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.403177 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.390600    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:57.511546 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.489097    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-102.ec2.internal\" not found" node="ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.511546 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.490478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.511546 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.491216    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:57.511546 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.494929    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-102.ec2.internal\" not found" node="ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.511546 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:57.497499    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:57.621510 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.591334    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:58.874152 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:57.691783    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:58.989867 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:58.931018    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.006700 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.003475    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 18:03:59.006700 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.003644    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 18:03:59.115138 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.031732    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.119405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.118816    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:58:58 +0000 UTC" deadline="2028-02-22 08:46:18.049223326 +0000 UTC"
Jul 09 18:03:59.119405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.118848    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14222h42m18.930381326s"
Jul 09 18:03:59.134123 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.132025    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.238022 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.141268    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:59.238022 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.232214    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.364245 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.332707    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.481499 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.433132    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.613323 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.533745    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.637190 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:03:59.634446    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-102.ec2.internal\" not found"
Jul 09 18:03:59.637190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.635653    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:59.740736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.655730    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:59.740736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.677815    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:59.740736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.692855    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:03:59.740736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.693873    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal"
Jul 09 18:03:59.740736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:03:59.712664    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:04:00.054927 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.054903    2567 apiserver.go:52] "Watching apiserver"
Jul 09 18:04:00.062001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.061976    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 18:04:00.062309 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.062288    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-w6gpm","kube-system/global-pull-secret-syncer-sbqht","kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gg6m4","openshift-image-registry/node-ca-9lw8l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal","openshift-multus/multus-8mh85","openshift-multus/multus-additional-cni-plugins-rxs4t","openshift-ovn-kubernetes/ovnkube-node-rpvm4","kube-system/konnectivity-agent-hkrz6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg","openshift-dns/node-resolver-rcdz4","openshift-multus/network-metrics-daemon-nm5xr","openshift-network-diagnostics/network-check-target-wpfsc"]
Jul 09 18:04:00.064035 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.064019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.065084 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.065064    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.065166 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.065146    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:00.069619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.069603    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:00.069715 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.069653    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:00.070941 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.070923    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.071218 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.071196    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 18:04:00.071325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.071312    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 18:04:00.071400 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.071385    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.071598 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.071580    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.071598 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.071591    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-flpst\""
Jul 09 18:04:00.072174 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.072159    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.073598 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.073583    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.074838 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.074818    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.076221 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.076204    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.077304 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.077290    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 18:04:00.077411 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.077395    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.077464 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.077448    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:00.078617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.078603    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.080068 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.080053    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.081728 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.081712    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.089515 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-cnibin\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.089601 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089519    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysconfig\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.089601 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysctl-d\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.089601 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/965f1315-c272-4861-9ab0-a1f343b6414f-agent-certs\") pod \"konnectivity-agent-hkrz6\" (UID: \"965f1315-c272-4861-9ab0-a1f343b6414f\") " pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.089601 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-cni-netd\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.089601 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089584    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.089761 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-os-release\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.089761 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-var-lib-kubelet\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.089761 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.089761 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089738    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hftd\" (UniqueName: \"kubernetes.io/projected/bcfc3b54-df61-4210-b823-da5675e01562-kube-api-access-4hftd\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.089894 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089763    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-run-ovn-kubernetes\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.089894 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-hostroot\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.089894 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089823    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-serviceca\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.089894 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089843    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-kubelet\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.089894 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-slash\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.089894 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-ovn\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-cni-bin\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-cni-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/965f1315-c272-4861-9ab0-a1f343b6414f-konnectivity-ca\") pod \"konnectivity-agent-hkrz6\" (UID: \"965f1315-c272-4861-9ab0-a1f343b6414f\") " pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-cnibin\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.089991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-systemd\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-system-cni-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090031    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-cni-bin\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090053    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-conf-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090081    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d58ddb01-ee8d-420a-8b95-96d197946d59-iptables-alerter-script\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zqdg\" (UniqueName: \"kubernetes.io/projected/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-kube-api-access-6zqdg\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090114    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wjf\" (UniqueName: \"kubernetes.io/projected/5daf7f63-4e19-423d-9361-06c07df0e551-kube-api-access-94wjf\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-log-socket\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-env-overrides\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbpvj\" (UniqueName: \"kubernetes.io/projected/42ad5776-8b11-47ee-a89a-b9fc896d5a49-kube-api-access-gbpvj\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090215    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41c442b1-6673-47d3-bc8f-38abe3b59588-kubelet-config\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-cni-multus\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-kubelet\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-tuned\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b80820-91f7-432a-981a-5b6f9b0792e0-tmp\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090318    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-system-cni-dir\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-etc-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-run\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-lib-modules\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-host\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090420    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsmn8\" (UniqueName: \"kubernetes.io/projected/d58ddb01-ee8d-420a-8b95-96d197946d59-kube-api-access-lsmn8\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-node-log\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovnkube-script-lib\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090473    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-modprobe-d\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090500    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-systemd\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-sys\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-daemon-config\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.090553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysctl-conf\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090594    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-run-netns\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41c442b1-6673-47d3-bc8f-38abe3b59588-dbus\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090670    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-cni-binary-copy\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-k8s-cni-cncf-io\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-netns\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-multus-certs\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-etc-kubernetes\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-cni-binary-copy\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090804    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-var-lib-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovnkube-config\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-socket-dir-parent\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2srq9\" (UniqueName: \"kubernetes.io/projected/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-kube-api-access-2srq9\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-kubernetes\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.091016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-os-release\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jhk4\" (UniqueName: \"kubernetes.io/projected/07b80820-91f7-432a-981a-5b6f9b0792e0-kube-api-access-4jhk4\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d58ddb01-ee8d-420a-8b95-96d197946d59-host-slash\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.090983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.091001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovn-node-metrics-cert\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.091024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.091042    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-host\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.091062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.091477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.091115    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-systemd-units\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.094602 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.094574    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.094682 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.094624    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.094736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.094714    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.094918 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.094903    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.094997 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.094979    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.095048 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.094989    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 18:04:00.095118 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.095088    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 18:04:00.095195 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.095179    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.096078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.095618    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 18:04:00.096078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.095671    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 18:04:00.096078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.095899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.096078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.095982    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h6pfk\""
Jul 09 18:04:00.096078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.096018    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fl585\""
Jul 09 18:04:00.096078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.096048    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7x97p\""
Jul 09 18:04:00.096359 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.096157    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 18:04:00.098194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.097190    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.098194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.097515    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 18:04:00.098194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.097769    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 18:04:00.099326 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.099309    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7trbd\""
Jul 09 18:04:00.099380 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.099331    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8dndp\""
Jul 09 18:04:00.099696 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.099667    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 18:04:00.099751 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.099703    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 18:04:00.099919 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.099896    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.100150 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.100138    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 18:04:00.100201 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.100141    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 18:04:00.100848 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.100828    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.100942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.100850    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 18:04:00.100942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.100828    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 18:04:00.101192 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.101176    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j49ws\""
Jul 09 18:04:00.101268 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.101205    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkwjl\""
Jul 09 18:04:00.101333 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.101318    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-58jcx\""
Jul 09 18:04:00.127869 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.127846    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:04:00.178849 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.178827    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 18:04:00.191393 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-run-netns\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.191516 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191425    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-run-netns\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.191516 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.191516 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41c442b1-6673-47d3-bc8f-38abe3b59588-dbus\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.191516 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191476    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.191516 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-cni-binary-copy\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191517    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-k8s-cni-cncf-io\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-netns\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-multus-certs\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-etc-kubernetes\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-cni-binary-copy\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-var-lib-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.191724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovnkube-config\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-socket-dir-parent\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2srq9\" (UniqueName: \"kubernetes.io/projected/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-kube-api-access-2srq9\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-kubernetes\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-os-release\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jhk4\" (UniqueName: \"kubernetes.io/projected/07b80820-91f7-432a-981a-5b6f9b0792e0-kube-api-access-4jhk4\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d58ddb01-ee8d-420a-8b95-96d197946d59-host-slash\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-registration-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-etc-kubernetes\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192050    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-var-lib-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-cni-binary-copy\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192143    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d58ddb01-ee8d-420a-8b95-96d197946d59-host-slash\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.191805    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-k8s-cni-cncf-io\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4nks\" (UniqueName: \"kubernetes.io/projected/cc25caa5-2d30-450f-958e-0c8b51386e5c-kube-api-access-p4nks\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-netns\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-kubernetes\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovn-node-metrics-cert\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-sys-fs\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-cni-binary-copy\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192289    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-host\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192151    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-socket-dir-parent\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-os-release\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-systemd-units\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-cnibin\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysconfig\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysctl-d\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/965f1315-c272-4861-9ab0-a1f343b6414f-agent-certs\") pod \"konnectivity-agent-hkrz6\" (UID: \"965f1315-c272-4861-9ab0-a1f343b6414f\") " pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.192490    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-cni-netd\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192545    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.193123 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.192555    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:00.692534053 +0000 UTC m=+3.039441961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-os-release\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192600    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-var-lib-kubelet\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192640    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-socket-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192667    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc25caa5-2d30-450f-958e-0c8b51386e5c-hosts-file\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192693    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41c442b1-6673-47d3-bc8f-38abe3b59588-dbus\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192746    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovnkube-config\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysconfig\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-systemd-units\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192872    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-cnibin\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192753    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hftd\" (UniqueName: \"kubernetes.io/projected/bcfc3b54-df61-4210-b823-da5675e01562-kube-api-access-4hftd\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192949    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysctl-d\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-var-lib-kubelet\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.192967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-run-ovn-kubernetes\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-hostroot\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.193942 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6lzw\" (UniqueName: \"kubernetes.io/projected/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-kube-api-access-j6lzw\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-serviceca\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-kubelet\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-slash\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-ovn\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-cni-bin\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-cni-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193197    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/965f1315-c272-4861-9ab0-a1f343b6414f-konnectivity-ca\") pod \"konnectivity-agent-hkrz6\" (UID: \"965f1315-c272-4861-9ab0-a1f343b6414f\") " pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-cnibin\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-systemd\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193249    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-system-cni-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-cni-bin\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-conf-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d58ddb01-ee8d-420a-8b95-96d197946d59-iptables-alerter-script\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-run-multus-certs\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zqdg\" (UniqueName: \"kubernetes.io/projected/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-kube-api-access-6zqdg\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193420    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-host\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193594    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-os-release\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.194826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193689    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193717    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-cni-netd\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.192614    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193767    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-cni-bin\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193769    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5daf7f63-4e19-423d-9361-06c07df0e551-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193829    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-run-ovn-kubernetes\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-systemd\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193888    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-cnibin\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-conf-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193910    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-run-ovn\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193936    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-slash\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-host-kubelet\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-cni-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.193983    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-system-cni-dir\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194030    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-hostroot\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-cni-bin\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194202    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-serviceca\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194251    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wjf\" (UniqueName: \"kubernetes.io/projected/5daf7f63-4e19-423d-9361-06c07df0e551-kube-api-access-94wjf\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.195721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-log-socket\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-env-overrides\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.194337    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:00.69431895 +0000 UTC m=+3.041226848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-log-socket\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbpvj\" (UniqueName: \"kubernetes.io/projected/42ad5776-8b11-47ee-a89a-b9fc896d5a49-kube-api-access-gbpvj\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41c442b1-6673-47d3-bc8f-38abe3b59588-kubelet-config\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-cni-multus\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-kubelet\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194545    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-tuned\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b80820-91f7-432a-981a-5b6f9b0792e0-tmp\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-system-cni-dir\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d58ddb01-ee8d-420a-8b95-96d197946d59-iptables-alerter-script\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41c442b1-6673-47d3-bc8f-38abe3b59588-kubelet-config\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-etc-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.196381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-cni-multus\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-run\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5daf7f63-4e19-423d-9361-06c07df0e551-system-cni-dir\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-lib-modules\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc25caa5-2d30-450f-958e-0c8b51386e5c-tmp-dir\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194773    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-run\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194804    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-host\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsmn8\" (UniqueName: \"kubernetes.io/projected/d58ddb01-ee8d-420a-8b95-96d197946d59-kube-api-access-lsmn8\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-node-log\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-lib-modules\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovnkube-script-lib\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-modprobe-d\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194931    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-host\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-systemd\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-node-log\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-sys\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-daemon-config\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.197190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysctl-conf\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-device-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195234    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-modprobe-d\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.194899    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-env-overrides\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-sys\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-sysctl-conf\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-systemd\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195431    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/965f1315-c272-4861-9ab0-a1f343b6414f-konnectivity-ca\") pod \"konnectivity-agent-hkrz6\" (UID: \"965f1315-c272-4861-9ab0-a1f343b6414f\") " pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195452    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-host-var-lib-kubelet\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195519    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovnkube-script-lib\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42ad5776-8b11-47ee-a89a-b9fc896d5a49-etc-openvswitch\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.195700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-multus-daemon-config\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.196854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/965f1315-c272-4861-9ab0-a1f343b6414f-agent-certs\") pod \"konnectivity-agent-hkrz6\" (UID: \"965f1315-c272-4861-9ab0-a1f343b6414f\") " pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.196689    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42ad5776-8b11-47ee-a89a-b9fc896d5a49-ovn-node-metrics-cert\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.197528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/07b80820-91f7-432a-981a-5b6f9b0792e0-etc-tuned\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.197955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.197578    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b80820-91f7-432a-981a-5b6f9b0792e0-tmp\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.209143 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.207364    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kd8b7"
Jul 09 18:04:00.213208 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.213184    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d903d9ca629bd1f13fc91f98e8c89a.slice/crio-bfc0064d91fd527ad3190db0ec52ac9dff7aa64b69ee71fcd2a67955a85db5f5 WatchSource:0}: Error finding container bfc0064d91fd527ad3190db0ec52ac9dff7aa64b69ee71fcd2a67955a85db5f5: Status 404 returned error can't find the container with id bfc0064d91fd527ad3190db0ec52ac9dff7aa64b69ee71fcd2a67955a85db5f5
Jul 09 18:04:00.218947 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.218930    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:04:00.219517 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.219464    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zqdg\" (UniqueName: \"kubernetes.io/projected/dd6e4bba-e65a-4097-9a87-4d6ca3ac163a-kube-api-access-6zqdg\") pod \"node-ca-9lw8l\" (UID: \"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a\") " pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.224349 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.224329    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbpvj\" (UniqueName: \"kubernetes.io/projected/42ad5776-8b11-47ee-a89a-b9fc896d5a49-kube-api-access-gbpvj\") pod \"ovnkube-node-rpvm4\" (UID: \"42ad5776-8b11-47ee-a89a-b9fc896d5a49\") " pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.224916 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.224901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jhk4\" (UniqueName: \"kubernetes.io/projected/07b80820-91f7-432a-981a-5b6f9b0792e0-kube-api-access-4jhk4\") pod \"tuned-gg6m4\" (UID: \"07b80820-91f7-432a-981a-5b6f9b0792e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.226239 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.226219    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2srq9\" (UniqueName: \"kubernetes.io/projected/e0a9c2d9-dc5c-48a1-86f5-1267d0c91227-kube-api-access-2srq9\") pod \"multus-8mh85\" (UID: \"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227\") " pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.227258 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.227239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hftd\" (UniqueName: \"kubernetes.io/projected/bcfc3b54-df61-4210-b823-da5675e01562-kube-api-access-4hftd\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.228691 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.228677    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kd8b7"
Jul 09 18:04:00.229195 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.229181    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:00.229239 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.229199    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:00.229239 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.229209    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:00.229314 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.229255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsmn8\" (UniqueName: \"kubernetes.io/projected/d58ddb01-ee8d-420a-8b95-96d197946d59-kube-api-access-lsmn8\") pod \"iptables-alerter-w6gpm\" (UID: \"d58ddb01-ee8d-420a-8b95-96d197946d59\") " pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.229314 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.229264    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:04:00.729250488 +0000 UTC m=+3.076158382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:00.231721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.231704    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wjf\" (UniqueName: \"kubernetes.io/projected/5daf7f63-4e19-423d-9361-06c07df0e551-kube-api-access-94wjf\") pod \"multus-additional-cni-plugins-rxs4t\" (UID: \"5daf7f63-4e19-423d-9361-06c07df0e551\") " pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.295852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.295823    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.295852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.295851    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.295868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc25caa5-2d30-450f-958e-0c8b51386e5c-tmp-dir\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.295954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.295986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-device-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296011    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-registration-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296039    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4nks\" (UniqueName: \"kubernetes.io/projected/cc25caa5-2d30-450f-958e-0c8b51386e5c-kube-api-access-p4nks\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296050    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-registration-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-sys-fs\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296088    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-sys-fs\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-device-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-socket-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc25caa5-2d30-450f-958e-0c8b51386e5c-hosts-file\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6lzw\" (UniqueName: \"kubernetes.io/projected/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-kube-api-access-j6lzw\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-socket-dir\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296268    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc25caa5-2d30-450f-958e-0c8b51386e5c-hosts-file\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.296399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.296266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc25caa5-2d30-450f-958e-0c8b51386e5c-tmp-dir\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.304588 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.304559    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4nks\" (UniqueName: \"kubernetes.io/projected/cc25caa5-2d30-450f-958e-0c8b51386e5c-kube-api-access-p4nks\") pod \"node-resolver-rcdz4\" (UID: \"cc25caa5-2d30-450f-958e-0c8b51386e5c\") " pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.304694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.304605    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6lzw\" (UniqueName: \"kubernetes.io/projected/4cfcb4fa-66ef-4431-9453-2ff4b7f52136-kube-api-access-j6lzw\") pod \"aws-ebs-csi-driver-node-m2hxg\" (UID: \"4cfcb4fa-66ef-4431-9453-2ff4b7f52136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.371625 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.371595    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c39ad98931262a830867440cbd12b9.slice/crio-445ef2a3214b3bdcd765721bb1985eb02196ef393d0ef656219db2ca26cf44eb WatchSource:0}: Error finding container 445ef2a3214b3bdcd765721bb1985eb02196ef393d0ef656219db2ca26cf44eb: Status 404 returned error can't find the container with id 445ef2a3214b3bdcd765721bb1985eb02196ef393d0ef656219db2ca26cf44eb
Jul 09 18:04:00.378168 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.378118    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:04:00.404432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.404398    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mh85"
Jul 09 18:04:00.411911 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.411883    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a9c2d9_dc5c_48a1_86f5_1267d0c91227.slice/crio-9d51b5e29e3e17897a69c71f1ef2698e72cd337b871f6181217744e9823cadba WatchSource:0}: Error finding container 9d51b5e29e3e17897a69c71f1ef2698e72cd337b871f6181217744e9823cadba: Status 404 returned error can't find the container with id 9d51b5e29e3e17897a69c71f1ef2698e72cd337b871f6181217744e9823cadba
Jul 09 18:04:00.413272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.413199    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gg6m4"
Jul 09 18:04:00.419936 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.419912    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b80820_91f7_432a_981a_5b6f9b0792e0.slice/crio-6989a1d37d27061c81381dc3fdf60bf0099e7af7c679bce4e6366789f204ea1e WatchSource:0}: Error finding container 6989a1d37d27061c81381dc3fdf60bf0099e7af7c679bce4e6366789f204ea1e: Status 404 returned error can't find the container with id 6989a1d37d27061c81381dc3fdf60bf0099e7af7c679bce4e6366789f204ea1e
Jul 09 18:04:00.424940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.424923    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9lw8l"
Jul 09 18:04:00.431847 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.431818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6e4bba_e65a_4097_9a87_4d6ca3ac163a.slice/crio-9ed0a980874eafdf53a999e068fb820a2fc697331376b069d0a29a8e8e6e8576 WatchSource:0}: Error finding container 9ed0a980874eafdf53a999e068fb820a2fc697331376b069d0a29a8e8e6e8576: Status 404 returned error can't find the container with id 9ed0a980874eafdf53a999e068fb820a2fc697331376b069d0a29a8e8e6e8576
Jul 09 18:04:00.438076 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.438060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rxs4t"
Jul 09 18:04:00.445285 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.445258    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5daf7f63_4e19_423d_9361_06c07df0e551.slice/crio-0716ccd55d5891ca6f622d149ab2cde63c38c4b7d69bd34a648fd9c3c268febb WatchSource:0}: Error finding container 0716ccd55d5891ca6f622d149ab2cde63c38c4b7d69bd34a648fd9c3c268febb: Status 404 returned error can't find the container with id 0716ccd55d5891ca6f622d149ab2cde63c38c4b7d69bd34a648fd9c3c268febb
Jul 09 18:04:00.452314 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.452294    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:00.459527 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.459506    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ad5776_8b11_47ee_a89a_b9fc896d5a49.slice/crio-1ac5691b6e41bf16549eb73f0a63595ec7e45531f3c15477401e2084bd44593e WatchSource:0}: Error finding container 1ac5691b6e41bf16549eb73f0a63595ec7e45531f3c15477401e2084bd44593e: Status 404 returned error can't find the container with id 1ac5691b6e41bf16549eb73f0a63595ec7e45531f3c15477401e2084bd44593e
Jul 09 18:04:00.470815 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.470779    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w6gpm"
Jul 09 18:04:00.477640 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.477619    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:00.477982 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.477959    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58ddb01_ee8d_420a_8b95_96d197946d59.slice/crio-cfb932411106f73335cbbf13ad790b9a42df43527a4f52df0bff48e84648f064 WatchSource:0}: Error finding container cfb932411106f73335cbbf13ad790b9a42df43527a4f52df0bff48e84648f064: Status 404 returned error can't find the container with id cfb932411106f73335cbbf13ad790b9a42df43527a4f52df0bff48e84648f064
Jul 09 18:04:00.484706 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.484679    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965f1315_c272_4861_9ab0_a1f343b6414f.slice/crio-ac0226b88d798501371c0a3fc07a321fabd8d45806b427d22e43a445dba88f25 WatchSource:0}: Error finding container ac0226b88d798501371c0a3fc07a321fabd8d45806b427d22e43a445dba88f25: Status 404 returned error can't find the container with id ac0226b88d798501371c0a3fc07a321fabd8d45806b427d22e43a445dba88f25
Jul 09 18:04:00.493535 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.493508    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg"
Jul 09 18:04:00.501026 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.501001    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfcb4fa_66ef_4431_9453_2ff4b7f52136.slice/crio-79ffed1f6fd8f2bdddb9e850516120f1c30bf4f6861847d026c0f392c2246d6e WatchSource:0}: Error finding container 79ffed1f6fd8f2bdddb9e850516120f1c30bf4f6861847d026c0f392c2246d6e: Status 404 returned error can't find the container with id 79ffed1f6fd8f2bdddb9e850516120f1c30bf4f6861847d026c0f392c2246d6e
Jul 09 18:04:00.507065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.507047    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rcdz4"
Jul 09 18:04:00.514191 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:00.514170    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc25caa5_2d30_450f_958e_0c8b51386e5c.slice/crio-757c7d61bdbcadf29b0df871e54e098480f41f98cc41ede52ecd82f9e99eaed7 WatchSource:0}: Error finding container 757c7d61bdbcadf29b0df871e54e098480f41f98cc41ede52ecd82f9e99eaed7: Status 404 returned error can't find the container with id 757c7d61bdbcadf29b0df871e54e098480f41f98cc41ede52ecd82f9e99eaed7
Jul 09 18:04:00.698494 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.698470    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:00.698634 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.698505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:00.698634 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.698616    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:00.698722 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.698621    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:00.698722 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.698668    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:01.698656408 +0000 UTC m=+4.045564315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:00.698722 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.698718    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:01.698704228 +0000 UTC m=+4.045612126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:00.798900 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:00.798867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:00.799102 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.799076    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:00.799224 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.799108    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:00.799224 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.799121    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:00.799224 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:00.799182    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:04:01.79916408 +0000 UTC m=+4.146071993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:01.224463 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.224224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rcdz4" event={"ID":"cc25caa5-2d30-450f-958e-0c8b51386e5c","Type":"ContainerStarted","Data":"757c7d61bdbcadf29b0df871e54e098480f41f98cc41ede52ecd82f9e99eaed7"}
Jul 09 18:04:01.231366 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.231329    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:59:00 +0000 UTC" deadline="2028-01-16 10:15:23.264197703 +0000 UTC"
Jul 09 18:04:01.231366 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.231366    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13336h11m22.032836595s"
Jul 09 18:04:01.240563 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.240528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg" event={"ID":"4cfcb4fa-66ef-4431-9453-2ff4b7f52136","Type":"ContainerStarted","Data":"79ffed1f6fd8f2bdddb9e850516120f1c30bf4f6861847d026c0f392c2246d6e"}
Jul 09 18:04:01.255944 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.255911    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w6gpm" event={"ID":"d58ddb01-ee8d-420a-8b95-96d197946d59","Type":"ContainerStarted","Data":"cfb932411106f73335cbbf13ad790b9a42df43527a4f52df0bff48e84648f064"}
Jul 09 18:04:01.273144 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.273104    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"1ac5691b6e41bf16549eb73f0a63595ec7e45531f3c15477401e2084bd44593e"}
Jul 09 18:04:01.292905 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.292862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerStarted","Data":"0716ccd55d5891ca6f622d149ab2cde63c38c4b7d69bd34a648fd9c3c268febb"}
Jul 09 18:04:01.305631 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.305573    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gg6m4" event={"ID":"07b80820-91f7-432a-981a-5b6f9b0792e0","Type":"ContainerStarted","Data":"6989a1d37d27061c81381dc3fdf60bf0099e7af7c679bce4e6366789f204ea1e"}
Jul 09 18:04:01.315830 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.315689    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mh85" event={"ID":"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227","Type":"ContainerStarted","Data":"9d51b5e29e3e17897a69c71f1ef2698e72cd337b871f6181217744e9823cadba"}
Jul 09 18:04:01.332836 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.332718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal" event={"ID":"03c39ad98931262a830867440cbd12b9","Type":"ContainerStarted","Data":"445ef2a3214b3bdcd765721bb1985eb02196ef393d0ef656219db2ca26cf44eb"}
Jul 09 18:04:01.352114 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.352049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkrz6" event={"ID":"965f1315-c272-4861-9ab0-a1f343b6414f","Type":"ContainerStarted","Data":"ac0226b88d798501371c0a3fc07a321fabd8d45806b427d22e43a445dba88f25"}
Jul 09 18:04:01.368425 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.368370    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9lw8l" event={"ID":"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a","Type":"ContainerStarted","Data":"9ed0a980874eafdf53a999e068fb820a2fc697331376b069d0a29a8e8e6e8576"}
Jul 09 18:04:01.379348 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.379309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal" event={"ID":"33d903d9ca629bd1f13fc91f98e8c89a","Type":"ContainerStarted","Data":"bfc0064d91fd527ad3190db0ec52ac9dff7aa64b69ee71fcd2a67955a85db5f5"}
Jul 09 18:04:01.707154 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.707106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:01.707343 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.707163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:01.707343 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.707247    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:01.707343 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.707271    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:01.707343 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.707324    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:03.707303524 +0000 UTC m=+6.054211430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:01.707343 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.707344    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:03.707333721 +0000 UTC m=+6.054241620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:01.808111 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:01.808071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:01.808301 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.808283    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:01.808376 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.808314    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:01.808376 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.808329    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:01.808480 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:01.808388    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:04:03.808370405 +0000 UTC m=+6.155278323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:02.223889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:02.223862    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:02.224068 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:02.223979    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:02.224194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:02.224171    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:02.224324 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:02.224278    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:02.224380 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:02.224333    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:02.224433 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:02.224403    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:02.231808 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:02.231545    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:59:00 +0000 UTC" deadline="2028-04-19 04:47:58.313566502 +0000 UTC"
Jul 09 18:04:02.231808 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:02.231575    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15586h43m56.081995469s"
Jul 09 18:04:03.726297 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:03.726216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:03.726750 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:03.726319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:03.726750 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.726504    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:03.726750 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.726573    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:07.72655248 +0000 UTC m=+10.073460374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:03.727006 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.726988    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:03.727081 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.727042    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:07.727027328 +0000 UTC m=+10.073935221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:03.827598 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:03.827560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:03.827771 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.827754    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:03.827876 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.827794    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:03.827876 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.827809    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:03.827876 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:03.827865    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:04:07.827847379 +0000 UTC m=+10.174755292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:04.189720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:04.189688    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:04.189899 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:04.189728    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:04.189899 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:04.189693    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:04.189899 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:04.189852    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:04.190058 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:04.189999    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:04.190117 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:04.190090    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:06.189421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:06.189306    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:06.189421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:06.189312    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:06.189921 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:06.189428    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:06.189921 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:06.189465    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:06.189921 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:06.189575    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:06.189921 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:06.189663    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:07.771207 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:07.771170    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:07.771642 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:07.771222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:07.771642 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.771335    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:07.771642 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.771357    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:07.771642 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.771418    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:15.771398932 +0000 UTC m=+18.118306830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:07.771642 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.771437    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:15.771427266 +0000 UTC m=+18.118335159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:07.872285 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:07.872247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:07.872467 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.872438    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:07.872467 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.872464    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:07.872571 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.872477    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:07.872571 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:07.872540    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:04:15.872521328 +0000 UTC m=+18.219429236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:08.190121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:08.190086    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:08.190121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:08.190120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:08.190383 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:08.190214    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:08.190589 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:08.190568    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:08.190850 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:08.190717    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:08.190850 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:08.190819    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:10.189184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:10.188978    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:10.189184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:10.189014    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:10.189184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:10.189108    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:10.189184 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:10.189121    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:10.189770 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:10.189217    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:10.189770 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:10.189305    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:12.188540 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:12.188510    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:12.188994 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:12.188548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:12.188994 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:12.188520    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:12.188994 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:12.188613    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:12.188994 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:12.188685    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:12.188994 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:12.188808    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:14.188671 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:14.188630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:14.189146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:14.188637    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:14.189146 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:14.188797    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:14.189146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:14.188837    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:14.189146 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:14.188936    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:14.189146 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:14.189000    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:15.831833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:15.831779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:15.832283 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:15.831857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:15.832283 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.831926    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:15.832283 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.832004    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:31.831986578 +0000 UTC m=+34.178894491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:15.832283 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.832023    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:15.832283 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.832084    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:31.832069553 +0000 UTC m=+34.178977447 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:15.933010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:15.932977    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:15.933182 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.933148    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:15.933182 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.933172    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:15.933261 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.933186    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:15.933261 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:15.933246    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:04:31.933229444 +0000 UTC m=+34.280137357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:16.188824 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:16.188714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:16.188824 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:16.188714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:16.189042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:16.188857    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:16.189042 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:16.188875    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:16.189042 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:16.188929    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:16.189042 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:16.189009    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:18.189779 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:18.189746    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:18.190233 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:18.189866    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:18.190233 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:18.189871    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:18.190233 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:18.189941    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:18.190233 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:18.189962    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:18.190233 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:18.190021    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:20.189321 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:20.189282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:20.189775 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:20.189285    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:20.189775 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:20.189402    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:20.189775 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:20.189492    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:20.189775 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:20.189290    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:20.189775 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:20.189577    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:21.430473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.430241    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:04:21.430870 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.430844    2567 generic.go:356] "Generic (PLEG): container finished" podID="42ad5776-8b11-47ee-a89a-b9fc896d5a49" containerID="f17ec47ea325e2dd32b8e364cd4f5503dd83c466f0126e811831abfd6de9811d" exitCode=1
Jul 09 18:04:21.430985 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.430910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"ff1c4b67bf9eac31f4cf7a7d425bac468f3faf379ae60431a97ceb95c3877cba"}
Jul 09 18:04:21.431039 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.431011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"a79c8db4d7855a93b6cf9af691fee28d4397fd3df479e1684509329fb153ef00"}
Jul 09 18:04:21.431090 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.431061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"a503c00f294ae47f845b6b365eee74c43cf2f2edb911c0eca48fcf3f13b08bdf"}
Jul 09 18:04:21.431137 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.431109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerDied","Data":"f17ec47ea325e2dd32b8e364cd4f5503dd83c466f0126e811831abfd6de9811d"}
Jul 09 18:04:21.431181 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.431141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"09cb4672f96eac3a8bb1852b74be64ad563c13777d57a7845d805da2bf81a3de"}
Jul 09 18:04:21.432887 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.432710    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gg6m4" event={"ID":"07b80820-91f7-432a-981a-5b6f9b0792e0","Type":"ContainerStarted","Data":"f088227f785d37169eefa25a914278ab621d863a1bd4aad48bb8e7c03f3aa4d3"}
Jul 09 18:04:21.434732 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.434666    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mh85" event={"ID":"e0a9c2d9-dc5c-48a1-86f5-1267d0c91227","Type":"ContainerStarted","Data":"132cbb4138bad2c653eca0d4663967eb123cbd0fb6d5710b3881565f24c01bc5"}
Jul 09 18:04:21.436636 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.436432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal" event={"ID":"33d903d9ca629bd1f13fc91f98e8c89a","Type":"ContainerStarted","Data":"4a57d4e55282ba08a83d75360143e229d51497a52df5c0f8cbfced4a2287e5e9"}
Jul 09 18:04:21.472103 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.472054    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gg6m4" podStartSLOduration=3.3503445640000002 podStartE2EDuration="23.47204005s" totalImagesPullingTime="20.121695486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:21.471728707 +0000 UTC m=+23.818636624" watchObservedRunningTime="2026-07-09 18:04:21.47204005 +0000 UTC m=+23.818947962"
Jul 09 18:04:21.490777 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.490724    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8mh85" podStartSLOduration=3.324632286 podStartE2EDuration="23.490701412s" totalImagesPullingTime="20.166069126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:21.490693789 +0000 UTC m=+23.837601705" watchObservedRunningTime="2026-07-09 18:04:21.490701412 +0000 UTC m=+23.837609332"
Jul 09 18:04:21.527552 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:21.527507    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-102.ec2.internal" podStartSLOduration=22.527490346 podStartE2EDuration="22.527490346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:21.520829926 +0000 UTC m=+23.867737838" watchObservedRunningTime="2026-07-09 18:04:21.527490346 +0000 UTC m=+23.874398262"
Jul 09 18:04:22.189405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.189154    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:22.189642 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.189154    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:22.189642 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:22.189516    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:22.189642 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.189241    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:22.189642 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:22.189607    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:22.189929 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:22.189668    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:22.440904 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.440776    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w6gpm" event={"ID":"d58ddb01-ee8d-420a-8b95-96d197946d59","Type":"ContainerStarted","Data":"39938ec994be6a9cff7b46782c34cd5f6d48f766b3082051caf29f1816d1fedd"}
Jul 09 18:04:22.444162 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.444096    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:04:22.444493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.444457    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"6e13d4b81e66e3d347b0770972961e4c02896604a4f6c1536c098fcf8bae7885"}
Jul 09 18:04:22.445815 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.445772    2567 generic.go:356] "Generic (PLEG): container finished" podID="5daf7f63-4e19-423d-9361-06c07df0e551" containerID="0f1df980c95c3a88dcf9dfb050ee449b3b4a2d98d81ab30ba27f724fbbcf8d29" exitCode=0
Jul 09 18:04:22.445925 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.445818    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerDied","Data":"0f1df980c95c3a88dcf9dfb050ee449b3b4a2d98d81ab30ba27f724fbbcf8d29"}
Jul 09 18:04:22.449769 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.449750    2567 generic.go:356] "Generic (PLEG): container finished" podID="03c39ad98931262a830867440cbd12b9" containerID="aea6980c46ced466c54ef877c6009515b32ab5f4bb1f2f369b2e14220874aaa4" exitCode=0
Jul 09 18:04:22.449887 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.449847    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal" event={"ID":"03c39ad98931262a830867440cbd12b9","Type":"ContainerDied","Data":"aea6980c46ced466c54ef877c6009515b32ab5f4bb1f2f369b2e14220874aaa4"}
Jul 09 18:04:22.451339 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.451317    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkrz6" event={"ID":"965f1315-c272-4861-9ab0-a1f343b6414f","Type":"ContainerStarted","Data":"94269c9dc7b180e133bd993121684945c7c5024d7a567aaff3d61028ff343c65"}
Jul 09 18:04:22.452632 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.452612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9lw8l" event={"ID":"dd6e4bba-e65a-4097-9a87-4d6ca3ac163a","Type":"ContainerStarted","Data":"629cd0fa6e0e10152f14e217877dde357177be4efb79c2b46b609a9c9bbc00d5"}
Jul 09 18:04:22.453834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.453811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rcdz4" event={"ID":"cc25caa5-2d30-450f-958e-0c8b51386e5c","Type":"ContainerStarted","Data":"20804864beaf297473ede09b83e62708bab11a6bec271890ea6822d3f647dcf2"}
Jul 09 18:04:22.455468 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.455447    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg" event={"ID":"4cfcb4fa-66ef-4431-9453-2ff4b7f52136","Type":"ContainerStarted","Data":"ef517f4dcb872c8b4ae0f4068c63d55bf85368bdf831c620f427954bbc007b72"}
Jul 09 18:04:22.458502 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.458468    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w6gpm" podStartSLOduration=4.413898453 podStartE2EDuration="24.458457124s" totalImagesPullingTime="20.044558671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:22.458239296 +0000 UTC m=+24.805147229" watchObservedRunningTime="2026-07-09 18:04:22.458457124 +0000 UTC m=+24.805365040"
Jul 09 18:04:22.494694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.494645    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hkrz6" podStartSLOduration=4.456860617 podStartE2EDuration="24.494629084s" totalImagesPullingTime="20.037768467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:22.494394284 +0000 UTC m=+24.841302201" watchObservedRunningTime="2026-07-09 18:04:22.494629084 +0000 UTC m=+24.841536997"
Jul 09 18:04:22.542736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.542676    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9lw8l" podStartSLOduration=4.434751583 podStartE2EDuration="24.54265676s" totalImagesPullingTime="20.107905177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:22.527738345 +0000 UTC m=+24.874646277" watchObservedRunningTime="2026-07-09 18:04:22.54265676 +0000 UTC m=+24.889564677"
Jul 09 18:04:22.543382 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.543345    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rcdz4" podStartSLOduration=4.517690404 podStartE2EDuration="24.543335209s" totalImagesPullingTime="20.025644805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:22.542311875 +0000 UTC m=+24.889219813" watchObservedRunningTime="2026-07-09 18:04:22.543335209 +0000 UTC m=+24.890243126"
Jul 09 18:04:22.784042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:22.784013    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 18:04:23.176072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.175933    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:04:22.784034351Z","UUID":"8feb984d-3252-4c16-8fcc-0e5eefec97a8","Handler":null,"Name":"","Endpoint":""}
Jul 09 18:04:23.177613 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.177590    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 18:04:23.177766 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.177636    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 18:04:23.460569 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.460529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal" event={"ID":"03c39ad98931262a830867440cbd12b9","Type":"ContainerStarted","Data":"35a90d3eb24b5b1c464d77fb0ab2253f3e5cdf388b16dfc0ac944cecb3ba0e1f"}
Jul 09 18:04:23.462595 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.462564    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg" event={"ID":"4cfcb4fa-66ef-4431-9453-2ff4b7f52136","Type":"ContainerStarted","Data":"32c1b9a50d5cb16cda7e4cbdde3dcd5a906fa85da42d82f435cd4201c18ded37"}
Jul 09 18:04:23.479037 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.478983    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-102.ec2.internal" podStartSLOduration=24.478961461 podStartE2EDuration="24.478961461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:23.478827644 +0000 UTC m=+25.825735587" watchObservedRunningTime="2026-07-09 18:04:23.478961461 +0000 UTC m=+25.825869378"
Jul 09 18:04:23.676446 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.676416    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:23.677039 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:23.677017    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:24.189036 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.188949    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:24.189190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.188949    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:24.189190 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:24.189078    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:24.189279 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:24.189195    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:24.189279 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.188949    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:24.189359 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:24.189286    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:24.467995 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.467969    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:04:24.468412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.468293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"9698dc30d01e9bee935a0458ee00d913080d67f2517ebd09ce05e7513473eb98"}
Jul 09 18:04:24.468460 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.468445    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:24.469010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:24.468993    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hkrz6"
Jul 09 18:04:25.474046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:25.474004    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg" event={"ID":"4cfcb4fa-66ef-4431-9453-2ff4b7f52136","Type":"ContainerStarted","Data":"77bf53993ebcc389cbea13b1be1fbd6371ced723f1f0178ec6d630ba2ea4ff31"}
Jul 09 18:04:25.499058 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:25.499001    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2hxg" podStartSLOduration=4.695261636 podStartE2EDuration="27.49898168s" totalImagesPullingTime="22.803720044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:25.498327028 +0000 UTC m=+27.845234971" watchObservedRunningTime="2026-07-09 18:04:25.49898168 +0000 UTC m=+27.845889598"
Jul 09 18:04:26.189232 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:26.189174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:26.189393 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:26.189174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:26.189393 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:26.189313    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:26.189393 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:26.189348    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:26.189566 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:26.189460    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:26.189566 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:26.189547    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:27.481754 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.481568    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:04:27.482530 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.482096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"787959a525fe57b51aac55fe10d8437e0954cfe17ca2ec34fde4c7f7204a05cf"}
Jul 09 18:04:27.482530 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.482374    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:27.482649 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.482628    2567 scope.go:117] "RemoveContainer" containerID="f17ec47ea325e2dd32b8e364cd4f5503dd83c466f0126e811831abfd6de9811d"
Jul 09 18:04:27.484135 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.484111    2567 generic.go:356] "Generic (PLEG): container finished" podID="5daf7f63-4e19-423d-9361-06c07df0e551" containerID="05ab1ebc4e97f96e18017c384f77e77f45ceca1987bb45fc80a021b321a4b46c" exitCode=0
Jul 09 18:04:27.484267 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.484151    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerDied","Data":"05ab1ebc4e97f96e18017c384f77e77f45ceca1987bb45fc80a021b321a4b46c"}
Jul 09 18:04:27.498331 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:27.498312    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:28.190480 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.190270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:28.190480 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:28.190388    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:28.190865 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.190841    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:28.190993 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.190877    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:28.190993 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:28.190941    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:28.191106 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:28.191035    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:28.491049 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.491028    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:04:28.491438 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.491416    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" event={"ID":"42ad5776-8b11-47ee-a89a-b9fc896d5a49","Type":"ContainerStarted","Data":"9f336c8d7594ff0a8549b81cda8c9818405b3ac15af4a70182f364ba2970572e"}
Jul 09 18:04:28.491796 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.491764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:28.491865 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.491843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:28.515125 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.515100    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:04:28.538875 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.538711    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4" podStartSLOduration=10.055404641 podStartE2EDuration="30.538693064s" totalImagesPullingTime="20.483288423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:28.537469498 +0000 UTC m=+30.884377439" watchObservedRunningTime="2026-07-09 18:04:28.538693064 +0000 UTC m=+30.885600995"
Jul 09 18:04:28.904599 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.904514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sbqht"]
Jul 09 18:04:28.904756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.904670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:28.904858 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:28.904780    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:28.907143 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.907116    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nm5xr"]
Jul 09 18:04:28.907245 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.907215    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:28.907312 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:28.907293    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:28.915801 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.915766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wpfsc"]
Jul 09 18:04:28.915882 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:28.915861    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:28.915939 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:28.915924    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:29.495797 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:29.495762    2567 generic.go:356] "Generic (PLEG): container finished" podID="5daf7f63-4e19-423d-9361-06c07df0e551" containerID="66b1688b96adbd2f7dd0a57e9e7f533a3bbdf70ce5b3d3bb2e498f041b808a4d" exitCode=0
Jul 09 18:04:29.496328 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:29.495852    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerDied","Data":"66b1688b96adbd2f7dd0a57e9e7f533a3bbdf70ce5b3d3bb2e498f041b808a4d"}
Jul 09 18:04:30.189288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:30.189266    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:30.189440 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:30.189315    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:30.189440 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:30.189401    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:30.189553 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:30.189502    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:31.188569 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:31.188405    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:31.188939 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.188633    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:31.502628 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:31.502600    2567 generic.go:356] "Generic (PLEG): container finished" podID="5daf7f63-4e19-423d-9361-06c07df0e551" containerID="745454849c3a05e3b72caaae8a3dafb50e2066ddd18f59dbfbf78d0c3438f4d4" exitCode=0
Jul 09 18:04:31.502816 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:31.502634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerDied","Data":"745454849c3a05e3b72caaae8a3dafb50e2066ddd18f59dbfbf78d0c3438f4d4"}
Jul 09 18:04:31.853367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:31.853288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:31.853367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:31.853323    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:31.853538 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.853429    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:31.853538 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.853482    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret podName:41c442b1-6673-47d3-bc8f-38abe3b59588 nodeName:}" failed. No retries permitted until 2026-07-09 18:05:03.853468327 +0000 UTC m=+66.200376221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret") pod "global-pull-secret-syncer-sbqht" (UID: "41c442b1-6673-47d3-bc8f-38abe3b59588") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:31.853538 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.853429    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:31.853631 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.853565    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:05:03.853552044 +0000 UTC m=+66.200459948 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:31.953684 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:31.953647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:31.953868 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.953848    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:31.953913 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.953873    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:31.953913 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.953887    2567 projected.go:196] Error preparing data for projected volume kube-api-access-45wtm for pod openshift-network-diagnostics/network-check-target-wpfsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:31.953972 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:31.953948    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm podName:28fe1fbc-df12-44ba-ab2c-f8ba3c306afe nodeName:}" failed. No retries permitted until 2026-07-09 18:05:03.9539298 +0000 UTC m=+66.300837701 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-45wtm" (UniqueName: "kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm") pod "network-check-target-wpfsc" (UID: "28fe1fbc-df12-44ba-ab2c-f8ba3c306afe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:32.188483 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:32.188405    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:32.188483 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:32.188455    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:32.189030 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:32.188545    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sbqht" podUID="41c442b1-6673-47d3-bc8f-38abe3b59588"
Jul 09 18:04:32.189030 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:32.188686    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:04:33.188453 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.188422    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:33.188668 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:33.188542    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpfsc" podUID="28fe1fbc-df12-44ba-ab2c-f8ba3c306afe"
Jul 09 18:04:33.418887 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.418862    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-102.ec2.internal" event="NodeReady"
Jul 09 18:04:33.419048 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.418993    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 18:04:33.456255 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.456028    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f54f75b7c-874l5"]
Jul 09 18:04:33.479190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.478570    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lctck"]
Jul 09 18:04:33.479190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.478621    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.481279 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.481249    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 18:04:33.481567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.481549    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 18:04:33.481956 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.481756    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 18:04:33.482696 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.482560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6mm8x\""
Jul 09 18:04:33.488747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.488722    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 18:04:33.503770 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.503750    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f54f75b7c-874l5"]
Jul 09 18:04:33.503881 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.503797    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6hnmt"]
Jul 09 18:04:33.504278 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.504248    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.507059 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.506915    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 18:04:33.507059 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.506951    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jd75b\""
Jul 09 18:04:33.507209 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.507163    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 18:04:33.528746 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.528729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lctck"]
Jul 09 18:04:33.528863 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.528765    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hnmt"]
Jul 09 18:04:33.528913 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.528881    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.531436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.531325    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 18:04:33.531436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.531350    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 18:04:33.531436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.531372    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lkb9p\""
Jul 09 18:04:33.531436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.531434    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 18:04:33.567575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-ca-trust-extracted\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567697 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpzs4\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-kube-api-access-qpzs4\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567697 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567669    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-certificates\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567819 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-image-registry-private-configuration\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567819 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567819 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-bound-sa-token\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567938 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-installation-pull-secrets\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.567988 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.567935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-trusted-ca\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.613698 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.613638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:33.668688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668655    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.668902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-trusted-ca\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.668902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668724    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m29s\" (UniqueName: \"kubernetes.io/projected/cb03f821-623c-4a50-841d-8642fa93b2ef-kube-api-access-8m29s\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.668902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-ca-trust-extracted\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.668902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0236ff24-a897-4d0f-b0b0-23d3a7904f2b-cert\") pod \"ingress-canary-6hnmt\" (UID: \"0236ff24-a897-4d0f-b0b0-23d3a7904f2b\") " pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.669084 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpzs4\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-kube-api-access-qpzs4\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669084 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.668990    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-certificates\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669084 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb03f821-623c-4a50-841d-8642fa93b2ef-config-volume\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.669084 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-image-registry-private-configuration\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669261 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669261 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-bound-sa-token\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669261 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669231    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-ca-trust-extracted\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669261 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:33.669241    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:04:33.669261 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:33.669256    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:04:33.669261 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb03f821-623c-4a50-841d-8642fa93b2ef-tmp-dir\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.669515 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669296    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgbmt\" (UniqueName: \"kubernetes.io/projected/0236ff24-a897-4d0f-b0b0-23d3a7904f2b-kube-api-access-lgbmt\") pod \"ingress-canary-6hnmt\" (UID: \"0236ff24-a897-4d0f-b0b0-23d3a7904f2b\") " pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.669515 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:33.669308    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:34.169289992 +0000 UTC m=+36.516197905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:04:33.669515 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-installation-pull-secrets\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669730 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669701    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-trusted-ca\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.669864 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.669747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-certificates\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.673719 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.673697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-installation-pull-secrets\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.673719 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.673713    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-image-registry-private-configuration\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.680191 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.680172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-bound-sa-token\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.680461 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.680437    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpzs4\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-kube-api-access-qpzs4\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:33.699734 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.699708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:33.770301 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770218    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb03f821-623c-4a50-841d-8642fa93b2ef-tmp-dir\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.770301 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgbmt\" (UniqueName: \"kubernetes.io/projected/0236ff24-a897-4d0f-b0b0-23d3a7904f2b-kube-api-access-lgbmt\") pod \"ingress-canary-6hnmt\" (UID: \"0236ff24-a897-4d0f-b0b0-23d3a7904f2b\") " pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.770301 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.770550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m29s\" (UniqueName: \"kubernetes.io/projected/cb03f821-623c-4a50-841d-8642fa93b2ef-kube-api-access-8m29s\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.770550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0236ff24-a897-4d0f-b0b0-23d3a7904f2b-cert\") pod \"ingress-canary-6hnmt\" (UID: \"0236ff24-a897-4d0f-b0b0-23d3a7904f2b\") " pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.770550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb03f821-623c-4a50-841d-8642fa93b2ef-config-volume\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.770550 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:33.770463    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:04:33.770760 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:33.770586    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:04:34.27056751 +0000 UTC m=+36.617475423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:04:33.770760 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.770616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb03f821-623c-4a50-841d-8642fa93b2ef-tmp-dir\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.771114 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.771093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb03f821-623c-4a50-841d-8642fa93b2ef-config-volume\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.773205 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.773185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0236ff24-a897-4d0f-b0b0-23d3a7904f2b-cert\") pod \"ingress-canary-6hnmt\" (UID: \"0236ff24-a897-4d0f-b0b0-23d3a7904f2b\") " pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.779006 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.778986    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m29s\" (UniqueName: \"kubernetes.io/projected/cb03f821-623c-4a50-841d-8642fa93b2ef-kube-api-access-8m29s\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:33.780879 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.780862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgbmt\" (UniqueName: \"kubernetes.io/projected/0236ff24-a897-4d0f-b0b0-23d3a7904f2b-kube-api-access-lgbmt\") pod \"ingress-canary-6hnmt\" (UID: \"0236ff24-a897-4d0f-b0b0-23d3a7904f2b\") " pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.838718 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.838686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hnmt"
Jul 09 18:04:33.999322 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:33.999252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hnmt"]
Jul 09 18:04:34.174089 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.174009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:34.174243 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:34.174164    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:04:34.174243 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:34.174180    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:04:34.174243 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:34.174234    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:35.17421796 +0000 UTC m=+37.521125860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:04:34.189303 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.189271    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:04:34.189989 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.189276    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:04:34.192175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.191988    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:04:34.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.192050    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7k8ns\""
Jul 09 18:04:34.192310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.192052    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:04:34.275122 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.275086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:34.275279 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:34.275238    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:04:34.275338 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:34.275326    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:04:35.275303186 +0000 UTC m=+37.622211079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:04:34.536474 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:34.536393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:35.183214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.183173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:35.183425 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:35.183331    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:04:35.183425 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:35.183353    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:04:35.183425 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:35.183417    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:37.183400003 +0000 UTC m=+39.530307897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:04:35.189263 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.189242    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:04:35.191966 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.191942    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 18:04:35.192350 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.192005    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w9bx9\""
Jul 09 18:04:35.192350 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.192204    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 18:04:35.284569 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.284532    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:35.284746 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:35.284691    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:04:35.284833 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:35.284779    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:04:37.284757719 +0000 UTC m=+39.631665614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:04:35.452320 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:04:35.452290    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0236ff24_a897_4d0f_b0b0_23d3a7904f2b.slice/crio-1dbcc99f2a3768501369890179a85b08798dd1f5fdabca29058e70f01f50a69d WatchSource:0}: Error finding container 1dbcc99f2a3768501369890179a85b08798dd1f5fdabca29058e70f01f50a69d: Status 404 returned error can't find the container with id 1dbcc99f2a3768501369890179a85b08798dd1f5fdabca29058e70f01f50a69d
Jul 09 18:04:35.513289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:35.513246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hnmt" event={"ID":"0236ff24-a897-4d0f-b0b0-23d3a7904f2b","Type":"ContainerStarted","Data":"1dbcc99f2a3768501369890179a85b08798dd1f5fdabca29058e70f01f50a69d"}
Jul 09 18:04:36.519409 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:36.519369    2567 generic.go:356] "Generic (PLEG): container finished" podID="5daf7f63-4e19-423d-9361-06c07df0e551" containerID="cd9af67d0eff066f2d1c0235eef2038f191cc549777a4645ddbebc484273a0e2" exitCode=0
Jul 09 18:04:36.519888 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:36.519422    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerDied","Data":"cd9af67d0eff066f2d1c0235eef2038f191cc549777a4645ddbebc484273a0e2"}
Jul 09 18:04:37.199687 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:37.199638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:37.199924 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:37.199751    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:04:37.199924 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:37.199764    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:04:37.199924 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:37.199903    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:41.199886491 +0000 UTC m=+43.546794385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:04:37.300186 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:37.300139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:37.300361 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:37.300275    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:04:37.300361 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:37.300354    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:04:41.300334787 +0000 UTC m=+43.647242693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:04:37.525804 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:37.525687    2567 generic.go:356] "Generic (PLEG): container finished" podID="5daf7f63-4e19-423d-9361-06c07df0e551" containerID="2124277e5879877e3d377af32e54eac255356d5d65909cd95aaaf6d10fa30cb9" exitCode=0
Jul 09 18:04:37.525804 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:37.525763    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerDied","Data":"2124277e5879877e3d377af32e54eac255356d5d65909cd95aaaf6d10fa30cb9"}
Jul 09 18:04:38.530095 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.529864    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hnmt" event={"ID":"0236ff24-a897-4d0f-b0b0-23d3a7904f2b","Type":"ContainerStarted","Data":"d8b9617bf420a53583b2e9189934cd38beb34738f46256aaf88bd4d6fceb7d0f"}
Jul 09 18:04:38.532977 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.532952    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" event={"ID":"5daf7f63-4e19-423d-9361-06c07df0e551","Type":"ContainerStarted","Data":"16e1eea977a695c048e5220ccd3c0ac299b487b201a512e3f6a1917b74227bd7"}
Jul 09 18:04:38.575658 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.575603    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6hnmt" podStartSLOduration=2.951015644 podStartE2EDuration="5.575588384s" totalImagesPullingTime="2.62457274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:38.574124615 +0000 UTC m=+40.921032531" watchObservedRunningTime="2026-07-09 18:04:38.575588384 +0000 UTC m=+40.922496329"
Jul 09 18:04:38.619129 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.619076    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rxs4t" podStartSLOduration=10.005260334 podStartE2EDuration="40.619059664s" totalImagesPullingTime="30.61379933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:38.617942347 +0000 UTC m=+40.964850293" watchObservedRunningTime="2026-07-09 18:04:38.619059664 +0000 UTC m=+40.965967581"
Jul 09 18:04:38.742537 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.742464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:38.806535 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.806495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:38.886525 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:38.886482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:39.000737 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:39.000647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:41.231497 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:41.231456    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:41.232014 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:41.231566    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:04:41.232014 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:41.231578    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:04:41.232014 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:41.231629    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:49.23161437 +0000 UTC m=+51.578522265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:04:41.332268 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:41.332236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:41.332429 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:41.332371    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:04:41.332469 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:41.332431    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:04:49.33241682 +0000 UTC m=+51.679324713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:04:41.646141 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:41.646044    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:41.817957 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:41.817921    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:45.664511 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:45.664473    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:45.720792 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:45.720764    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:47.006416 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:47.006381    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:47.107091 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:47.107059    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:47.375381 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:47.375299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:47.476615 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:47.476582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:49.288015 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:49.287972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:04:49.288398 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:49.288084    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:04:49.288398 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:49.288095    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:04:49.288398 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:49.288180    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:05:05.288166174 +0000 UTC m=+67.635074068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:04:49.388721 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:49.388680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:04:49.388911 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:49.388883    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:04:49.388975 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:04:49.388964    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:05:05.388946876 +0000 UTC m=+67.735854774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:04:51.659243 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:51.659206    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:51.788852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:51.788810    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:51.926040 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:51.925953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:52.026978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:52.026939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:52.723978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:52.723947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:52.878315 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:52.878284    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:53.751846 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:04:53.751807    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:00.508795 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:00.508758    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rpvm4"
Jul 09 18:05:00.712227 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:00.712193    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:00.815874 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:00.815771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:01.053680 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:01.053649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:03.900862 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:03.900812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:05:03.900862 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:03.900863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:05:03.904066 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:03.904046    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:05:03.904168 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:03.904151    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:05:03.912072 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:03.912041    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 18:05:03.912186 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:03.912126    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:06:07.912109277 +0000 UTC m=+130.259017171 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : secret "metrics-daemon-secret" not found
Jul 09 18:05:03.915763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:03.915747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41c442b1-6673-47d3-bc8f-38abe3b59588-original-pull-secret\") pod \"global-pull-secret-syncer-sbqht\" (UID: \"41c442b1-6673-47d3-bc8f-38abe3b59588\") " pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:05:04.001855 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.001817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:05:04.004876 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.004852    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 18:05:04.014440 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.014415    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 18:05:04.025763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.025729    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45wtm\" (UniqueName: \"kubernetes.io/projected/28fe1fbc-df12-44ba-ab2c-f8ba3c306afe-kube-api-access-45wtm\") pod \"network-check-target-wpfsc\" (UID: \"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe\") " pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:05:04.201569 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.201540    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sbqht"
Jul 09 18:05:04.303527 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.303503    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w9bx9\""
Jul 09 18:05:04.311778 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.311751    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:05:04.352267 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.352232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sbqht"]
Jul 09 18:05:04.355020 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:05:04.354981    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c442b1_6673_47d3_bc8f_38abe3b59588.slice/crio-8421571cf64c6fc9484f12e751c52c185bed01b013dd4012f5c0380a89b63851 WatchSource:0}: Error finding container 8421571cf64c6fc9484f12e751c52c185bed01b013dd4012f5c0380a89b63851: Status 404 returned error can't find the container with id 8421571cf64c6fc9484f12e751c52c185bed01b013dd4012f5c0380a89b63851
Jul 09 18:05:04.433686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.433555    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wpfsc"]
Jul 09 18:05:04.436080 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:05:04.436041    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fe1fbc_df12_44ba_ab2c_f8ba3c306afe.slice/crio-76ab74aa362e52c70800c2d0b26968346e6c5153aae20909fdd43e4019290055 WatchSource:0}: Error finding container 76ab74aa362e52c70800c2d0b26968346e6c5153aae20909fdd43e4019290055: Status 404 returned error can't find the container with id 76ab74aa362e52c70800c2d0b26968346e6c5153aae20909fdd43e4019290055
Jul 09 18:05:04.443939 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.443917    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:04.595138 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.595046    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:04.596999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.596967    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wpfsc" event={"ID":"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe","Type":"ContainerStarted","Data":"76ab74aa362e52c70800c2d0b26968346e6c5153aae20909fdd43e4019290055"}
Jul 09 18:05:04.598077 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:04.598050    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sbqht" event={"ID":"41c442b1-6673-47d3-bc8f-38abe3b59588","Type":"ContainerStarted","Data":"8421571cf64c6fc9484f12e751c52c185bed01b013dd4012f5c0380a89b63851"}
Jul 09 18:05:05.313539 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:05.313415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:05:05.314018 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:05.313645    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:05:05.314018 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:05.313669    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:05:05.314018 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:05.313738    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:05:37.313720147 +0000 UTC m=+99.660628049 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:05:05.414621 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:05.414567    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:05:05.414829 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:05.414754    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:05:05.414891 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:05.414852    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:05:37.414828781 +0000 UTC m=+99.761736680 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:05:07.579693 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:07.579643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:07.641576 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:07.641528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:07.819452 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:07.819412    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:07.918659 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:07.918570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:09.282478 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.282439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:09.361692 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.361656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:09.615632 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.615547    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wpfsc" event={"ID":"28fe1fbc-df12-44ba-ab2c-f8ba3c306afe","Type":"ContainerStarted","Data":"83f09345921d38ae10f4402249a26f63ff8cfe0f002c2c7c8c9d59cf9fdce533"}
Jul 09 18:05:09.615808 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.615678    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:05:09.616993 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.616972    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sbqht" event={"ID":"41c442b1-6673-47d3-bc8f-38abe3b59588","Type":"ContainerStarted","Data":"daaff5081dff0ad158f64e65062f7ec47ae61c69101fe0d3d427a907cafb63e0"}
Jul 09 18:05:09.632358 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.632300    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wpfsc" podStartSLOduration=67.16607987 podStartE2EDuration="1m11.632285566s" totalImagesPullingTime="4.466205696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:09.631303629 +0000 UTC m=+71.978211544" watchObservedRunningTime="2026-07-09 18:05:09.632285566 +0000 UTC m=+71.979193481"
Jul 09 18:05:09.645238 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:09.645195    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sbqht" podStartSLOduration=67.09431614 podStartE2EDuration="1m11.645178891s" totalImagesPullingTime="4.550862751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:09.644636484 +0000 UTC m=+71.991544419" watchObservedRunningTime="2026-07-09 18:05:09.645178891 +0000 UTC m=+71.992086808"
Jul 09 18:05:29.813643 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:29.813510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:29.880978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:29.880942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:37.346699 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:37.346654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:05:37.347221 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:37.346830    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:05:37.347221 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:37.346849    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:05:37.347221 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:37.346918    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:06:41.346900582 +0000 UTC m=+163.693808476 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:05:37.447625 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:37.447578    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:05:37.447779 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:37.447737    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:05:37.447880 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:05:37.447860    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:06:41.447842126 +0000 UTC m=+163.794750025 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:05:40.622281 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:05:40.622253    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wpfsc"
Jul 09 18:06:07.961645 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:06:07.961597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:06:07.962161 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:07.961720    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 18:06:07.962161 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:07.961802    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:09.961766703 +0000 UTC m=+252.308674597 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : secret "metrics-daemon-secret" not found
Jul 09 18:06:36.493052 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:36.492998    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"
Jul 09 18:06:36.513230 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:36.513200    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lctck" podUID="cb03f821-623c-4a50-841d-8642fa93b2ef"
Jul 09 18:06:36.822217 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:06:36.822139    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:06:36.822217 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:06:36.822177    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lctck"
Jul 09 18:06:37.206506 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:37.206448    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:06:41.395608 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:06:41.395552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:06:41.396036 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:41.395694    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:06:41.396036 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:41.395706    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f54f75b7c-874l5: secret "image-registry-tls" not found
Jul 09 18:06:41.396036 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:41.395766    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls podName:0ac7d7af-a5b1-4814-8edb-8f3472e3beb8 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:43.395753045 +0000 UTC m=+285.742660939 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls") pod "image-registry-6f54f75b7c-874l5" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8") : secret "image-registry-tls" not found
Jul 09 18:06:41.496208 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:06:41.496158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:06:41.496370 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:41.496308    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:06:41.496407 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:06:41.496383    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls podName:cb03f821-623c-4a50-841d-8642fa93b2ef nodeName:}" failed. No retries permitted until 2026-07-09 18:08:43.4963659 +0000 UTC m=+285.843273813 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls") pod "dns-default-lctck" (UID: "cb03f821-623c-4a50-841d-8642fa93b2ef") : secret "dns-default-metrics-tls" not found
Jul 09 18:06:50.188489 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:06:50.188450    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:07:25.167915 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:07:25.167886    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:08.642062 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.642023    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"]
Jul 09 18:08:08.644073 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.644056    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"]
Jul 09 18:08:08.644208 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.644188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.645995 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.645975    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:08.646374 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.646354    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 09 18:08:08.648624 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.648607    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gw57t\""
Jul 09 18:08:08.648624 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.648617    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 09 18:08:08.648773 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.648632    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 18:08:08.648773 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.648635    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-lxjl9\""
Jul 09 18:08:08.648773 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.648654    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 18:08:08.648905 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.648865    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 09 18:08:08.649030 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.649016    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 09 18:08:08.654851 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.654832    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"]
Jul 09 18:08:08.662931 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.662910    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"]
Jul 09 18:08:08.706978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.706941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:08.707172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.706985    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.707172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.707036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6ls\" (UniqueName: \"kubernetes.io/projected/e6c97669-d87d-4e2f-b35c-c7a39853020c-kube-api-access-kb6ls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.707172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.707083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8bbaae80-06b4-43bf-92f1-def9f70426f4-nginx-conf\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:08.707172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.707109    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6c97669-d87d-4e2f-b35c-c7a39853020c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.754345 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.754307    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"]
Jul 09 18:08:08.756115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.756098    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"]
Jul 09 18:08:08.756238 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.756218    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.757801 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.757770    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lplzp"]
Jul 09 18:08:08.757904 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.757888    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:08.759713 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.759695    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.760188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.760170    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:08:08.760188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.760174    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 09 18:08:08.760480 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.760459    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 09 18:08:08.760686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.760672    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-st6bx\""
Jul 09 18:08:08.764581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764562    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 18:08:08.764694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764606    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-47fkm\""
Jul 09 18:08:08.764694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764607    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 18:08:08.764831 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764771    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 09 18:08:08.764831 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764821    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 18:08:08.764940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764850    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 18:08:08.764940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.764850    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 09 18:08:08.765270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.765248    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-v9slg\""
Jul 09 18:08:08.770083 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.770063    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:08:08.773680 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.773648    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"]
Jul 09 18:08:08.773779 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.773763    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 09 18:08:08.777510 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.777490    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 09 18:08:08.778197 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.778177    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lplzp"]
Jul 09 18:08:08.781760 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.781736    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"]
Jul 09 18:08:08.789081 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.789059    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"]
Jul 09 18:08:08.791190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.791174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"
Jul 09 18:08:08.793999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.793978    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xf299\""
Jul 09 18:08:08.794187 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.794173    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 09 18:08:08.794829 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.794814    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:08:08.802580 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.802557    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"]
Jul 09 18:08:08.808103 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:08.808204 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.808262 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:08.808212    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:08.808262 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:08.808214    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 18:08:08.808334 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:08.808262    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls podName:e6c97669-d87d-4e2f-b35c-c7a39853020c nodeName:}" failed. No retries permitted until 2026-07-09 18:08:09.308248802 +0000 UTC m=+251.655156697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2qkks" (UID: "e6c97669-d87d-4e2f-b35c-c7a39853020c") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:08.808334 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a189a918-de0e-49b0-a269-79ef6f699c06-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.808420 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6ls\" (UniqueName: \"kubernetes.io/projected/e6c97669-d87d-4e2f-b35c-c7a39853020c-kube-api-access-kb6ls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.808420 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:08.808376    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert podName:8bbaae80-06b4-43bf-92f1-def9f70426f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:09.308359937 +0000 UTC m=+251.655267835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hgdbm" (UID: "8bbaae80-06b4-43bf-92f1-def9f70426f4") : secret "networking-console-plugin-cert" not found
Jul 09 18:08:08.808491 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7g7d\" (UniqueName: \"kubernetes.io/projected/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-kube-api-access-g7g7d\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.808491 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808438    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a189a918-de0e-49b0-a269-79ef6f699c06-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.808491 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a189a918-de0e-49b0-a269-79ef6f699c06-tmp\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.808597 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a189a918-de0e-49b0-a269-79ef6f699c06-snapshots\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.808597 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:08.808597 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr655\" (UniqueName: \"kubernetes.io/projected/f9296204-be27-4bae-8c31-3c88a9c08433-kube-api-access-sr655\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:08.808692 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808594    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tmxb\" (UniqueName: \"kubernetes.io/projected/d09d07a2-835f-49b3-a92b-98d1ab552714-kube-api-access-9tmxb\") pod \"volume-data-source-validator-78495597f6-2hvxp\" (UID: \"d09d07a2-835f-49b3-a92b-98d1ab552714\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"
Jul 09 18:08:08.808692 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808617    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsgg\" (UniqueName: \"kubernetes.io/projected/a189a918-de0e-49b0-a269-79ef6f699c06-kube-api-access-zlsgg\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.808692 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8bbaae80-06b4-43bf-92f1-def9f70426f4-nginx-conf\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:08.808799 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6c97669-d87d-4e2f-b35c-c7a39853020c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.808799 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.808799 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808739    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a189a918-de0e-49b0-a269-79ef6f699c06-serving-cert\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.808799 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.808763    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.809324 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.809306    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8bbaae80-06b4-43bf-92f1-def9f70426f4-nginx-conf\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:08.809394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.809378    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6c97669-d87d-4e2f-b35c-c7a39853020c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.830331 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.830299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6ls\" (UniqueName: \"kubernetes.io/projected/e6c97669-d87d-4e2f-b35c-c7a39853020c-kube-api-access-kb6ls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:08.909439 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a189a918-de0e-49b0-a269-79ef6f699c06-serving-cert\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.909439 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.909677 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a189a918-de0e-49b0-a269-79ef6f699c06-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.909677 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909486    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7g7d\" (UniqueName: \"kubernetes.io/projected/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-kube-api-access-g7g7d\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.909677 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a189a918-de0e-49b0-a269-79ef6f699c06-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.909677 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a189a918-de0e-49b0-a269-79ef6f699c06-tmp\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.909677 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a189a918-de0e-49b0-a269-79ef6f699c06-snapshots\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.909677 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:08.910001 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:08.909680    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:08:08.910001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr655\" (UniqueName: \"kubernetes.io/projected/f9296204-be27-4bae-8c31-3c88a9c08433-kube-api-access-sr655\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:08.910001 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:08.909749    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls podName:f9296204-be27-4bae-8c31-3c88a9c08433 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:09.409732848 +0000 UTC m=+251.756640771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls") pod "cluster-samples-operator-65db858894-fzdlj" (UID: "f9296204-be27-4bae-8c31-3c88a9c08433") : secret "samples-operator-tls" not found
Jul 09 18:08:08.910001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tmxb\" (UniqueName: \"kubernetes.io/projected/d09d07a2-835f-49b3-a92b-98d1ab552714-kube-api-access-9tmxb\") pod \"volume-data-source-validator-78495597f6-2hvxp\" (UID: \"d09d07a2-835f-49b3-a92b-98d1ab552714\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"
Jul 09 18:08:08.910001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsgg\" (UniqueName: \"kubernetes.io/projected/a189a918-de0e-49b0-a269-79ef6f699c06-kube-api-access-zlsgg\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.910001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.909856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.910308 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.910186    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a189a918-de0e-49b0-a269-79ef6f699c06-tmp\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.910308 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.910204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a189a918-de0e-49b0-a269-79ef6f699c06-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.910308 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.910245    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a189a918-de0e-49b0-a269-79ef6f699c06-snapshots\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.910515 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.910493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a189a918-de0e-49b0-a269-79ef6f699c06-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.910660 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.910639    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.911949 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.911923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a189a918-de0e-49b0-a269-79ef6f699c06-serving-cert\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.912072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.912044    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.920982 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.920956    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7g7d\" (UniqueName: \"kubernetes.io/projected/9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc-kube-api-access-g7g7d\") pod \"kube-storage-version-migrator-operator-799cb7f958-q7z4l\" (UID: \"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:08.924186 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.924159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr655\" (UniqueName: \"kubernetes.io/projected/f9296204-be27-4bae-8c31-3c88a9c08433-kube-api-access-sr655\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:08.924573 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.924539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsgg\" (UniqueName: \"kubernetes.io/projected/a189a918-de0e-49b0-a269-79ef6f699c06-kube-api-access-zlsgg\") pod \"insights-operator-68d9dd94cf-lplzp\" (UID: \"a189a918-de0e-49b0-a269-79ef6f699c06\") " pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:08.924660 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.924542    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tmxb\" (UniqueName: \"kubernetes.io/projected/d09d07a2-835f-49b3-a92b-98d1ab552714-kube-api-access-9tmxb\") pod \"volume-data-source-validator-78495597f6-2hvxp\" (UID: \"d09d07a2-835f-49b3-a92b-98d1ab552714\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"
Jul 09 18:08:08.978918 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:08.978882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:09.066461 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.066425    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"
Jul 09 18:08:09.077584 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.077559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:09.079352 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.079334    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp"
Jul 09 18:08:09.099186 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.099162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"
Jul 09 18:08:09.215289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.215250    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l"]
Jul 09 18:08:09.218912 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:09.218880    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf0633c_0a0f_4937_91f9_b9adfcbf5ecc.slice/crio-c7529032366df77b3808e84cc790582d2d9a7dfef57b2bcd948eee4d94dd7e48 WatchSource:0}: Error finding container c7529032366df77b3808e84cc790582d2d9a7dfef57b2bcd948eee4d94dd7e48: Status 404 returned error can't find the container with id c7529032366df77b3808e84cc790582d2d9a7dfef57b2bcd948eee4d94dd7e48
Jul 09 18:08:09.230674 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.230644    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lplzp"]
Jul 09 18:08:09.234043 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:09.234012    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda189a918_de0e_49b0_a269_79ef6f699c06.slice/crio-c452dae48c257d2347a0403bc0fb11cb7c3ca0833dc0d0f2f88b8bbcaa82934c WatchSource:0}: Error finding container c452dae48c257d2347a0403bc0fb11cb7c3ca0833dc0d0f2f88b8bbcaa82934c: Status 404 returned error can't find the container with id c452dae48c257d2347a0403bc0fb11cb7c3ca0833dc0d0f2f88b8bbcaa82934c
Jul 09 18:08:09.247503 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.247481    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp"]
Jul 09 18:08:09.250545 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:09.250519    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09d07a2_835f_49b3_a92b_98d1ab552714.slice/crio-5cf9d8e67fe5cf089f069d61bce9abcf29a1fef6ea2bb190049482fee10b690a WatchSource:0}: Error finding container 5cf9d8e67fe5cf089f069d61bce9abcf29a1fef6ea2bb190049482fee10b690a: Status 404 returned error can't find the container with id 5cf9d8e67fe5cf089f069d61bce9abcf29a1fef6ea2bb190049482fee10b690a
Jul 09 18:08:09.313287 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.313246    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:09.313287 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.313288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:09.313482 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:09.313395    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:09.313482 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:09.313445    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls podName:e6c97669-d87d-4e2f-b35c-c7a39853020c nodeName:}" failed. No retries permitted until 2026-07-09 18:08:10.313429023 +0000 UTC m=+252.660336916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2qkks" (UID: "e6c97669-d87d-4e2f-b35c-c7a39853020c") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:09.313482 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:09.313468    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 18:08:09.313592 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:09.313532    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert podName:8bbaae80-06b4-43bf-92f1-def9f70426f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:10.313515557 +0000 UTC m=+252.660423474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hgdbm" (UID: "8bbaae80-06b4-43bf-92f1-def9f70426f4") : secret "networking-console-plugin-cert" not found
Jul 09 18:08:09.414161 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:09.414128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:09.414304 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:09.414238    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:08:09.414304 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:09.414288    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls podName:f9296204-be27-4bae-8c31-3c88a9c08433 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:10.414275475 +0000 UTC m=+252.761183368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls") pod "cluster-samples-operator-65db858894-fzdlj" (UID: "f9296204-be27-4bae-8c31-3c88a9c08433") : secret "samples-operator-tls" not found
Jul 09 18:08:10.018899 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.018857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:08:10.019306 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.019021    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 18:08:10.019306 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.019104    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs podName:bcfc3b54-df61-4210-b823-da5675e01562 nodeName:}" failed. No retries permitted until 2026-07-09 18:10:12.019079803 +0000 UTC m=+374.365987697 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs") pod "network-metrics-daemon-nm5xr" (UID: "bcfc3b54-df61-4210-b823-da5675e01562") : secret "metrics-daemon-secret" not found
Jul 09 18:08:10.039435 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.039395    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp" event={"ID":"d09d07a2-835f-49b3-a92b-98d1ab552714","Type":"ContainerStarted","Data":"5cf9d8e67fe5cf089f069d61bce9abcf29a1fef6ea2bb190049482fee10b690a"}
Jul 09 18:08:10.040554 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.040519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp" event={"ID":"a189a918-de0e-49b0-a269-79ef6f699c06","Type":"ContainerStarted","Data":"c452dae48c257d2347a0403bc0fb11cb7c3ca0833dc0d0f2f88b8bbcaa82934c"}
Jul 09 18:08:10.041648 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.041627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l" event={"ID":"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc","Type":"ContainerStarted","Data":"c7529032366df77b3808e84cc790582d2d9a7dfef57b2bcd948eee4d94dd7e48"}
Jul 09 18:08:10.321454 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.321351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:10.321454 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.321410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:10.321677 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.321531    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 18:08:10.321677 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.321552    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:10.321677 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.321614    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert podName:8bbaae80-06b4-43bf-92f1-def9f70426f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:12.321592991 +0000 UTC m=+254.668500896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hgdbm" (UID: "8bbaae80-06b4-43bf-92f1-def9f70426f4") : secret "networking-console-plugin-cert" not found
Jul 09 18:08:10.321677 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.321636    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls podName:e6c97669-d87d-4e2f-b35c-c7a39853020c nodeName:}" failed. No retries permitted until 2026-07-09 18:08:12.321626459 +0000 UTC m=+254.668534386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2qkks" (UID: "e6c97669-d87d-4e2f-b35c-c7a39853020c") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:10.422249 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:10.422208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:10.422435 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.422397    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:08:10.422496 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:10.422482    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls podName:f9296204-be27-4bae-8c31-3c88a9c08433 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:12.422460908 +0000 UTC m=+254.769368809 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls") pod "cluster-samples-operator-65db858894-fzdlj" (UID: "f9296204-be27-4bae-8c31-3c88a9c08433") : secret "samples-operator-tls" not found
Jul 09 18:08:12.049763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.049660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp" event={"ID":"d09d07a2-835f-49b3-a92b-98d1ab552714","Type":"ContainerStarted","Data":"b4db87f6f98ee1960e1decc6474d0e92c447892f9798224f70439da9652c6c64"}
Jul 09 18:08:12.051167 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.051136    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp" event={"ID":"a189a918-de0e-49b0-a269-79ef6f699c06","Type":"ContainerStarted","Data":"579cc78a8ba3d918bf32d626f08985c116471f717e540547c8e21353071f0a2f"}
Jul 09 18:08:12.052549 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.052526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l" event={"ID":"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc","Type":"ContainerStarted","Data":"498e8b8cee79d46be016a90a76acb593f0de015c78c1a164e354d01d03cf1e40"}
Jul 09 18:08:12.067226 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.067184    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-2hvxp" podStartSLOduration=1.5350839729999999 podStartE2EDuration="4.067170075s" totalImagesPullingTime="2.532086102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:12.066670512 +0000 UTC m=+254.413578419" watchObservedRunningTime="2026-07-09 18:08:12.067170075 +0000 UTC m=+254.414078000"
Jul 09 18:08:12.120686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.120617    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp" podStartSLOduration=1.5733779989999999 podStartE2EDuration="4.120592861s" totalImagesPullingTime="2.547214862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:12.085691202 +0000 UTC m=+254.432599151" watchObservedRunningTime="2026-07-09 18:08:12.120592861 +0000 UTC m=+254.467500778"
Jul 09 18:08:12.121191 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.121160    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l" podStartSLOduration=1.55191843 podStartE2EDuration="4.121151314s" totalImagesPullingTime="2.569232884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:12.118918574 +0000 UTC m=+254.465826509" watchObservedRunningTime="2026-07-09 18:08:12.121151314 +0000 UTC m=+254.468059233"
Jul 09 18:08:12.214247 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.214208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:12.314345 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.314247    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:12.338848 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.338812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:12.339008 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.338860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:12.339008 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:12.338956    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 18:08:12.339008 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:12.338976    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:12.339159 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:12.339036    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert podName:8bbaae80-06b4-43bf-92f1-def9f70426f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:16.339016092 +0000 UTC m=+258.685923991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hgdbm" (UID: "8bbaae80-06b4-43bf-92f1-def9f70426f4") : secret "networking-console-plugin-cert" not found
Jul 09 18:08:12.339159 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:12.339056    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls podName:e6c97669-d87d-4e2f-b35c-c7a39853020c nodeName:}" failed. No retries permitted until 2026-07-09 18:08:16.339046388 +0000 UTC m=+258.685954287 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2qkks" (UID: "e6c97669-d87d-4e2f-b35c-c7a39853020c") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:12.439815 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.439747    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:12.440016 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:12.439925    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:08:12.440016 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:12.440004    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls podName:f9296204-be27-4bae-8c31-3c88a9c08433 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:16.439986229 +0000 UTC m=+258.786894136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls") pod "cluster-samples-operator-65db858894-fzdlj" (UID: "f9296204-be27-4bae-8c31-3c88a9c08433") : secret "samples-operator-tls" not found
Jul 09 18:08:12.624094 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.624000    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:12.726130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:12.726092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:13.344867 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.344834    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rjxn2"]
Jul 09 18:08:13.347269 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.347246    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.351247 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.351223    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 18:08:13.351379 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.351253    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 18:08:13.351654 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.351638    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lw2dl\""
Jul 09 18:08:13.364469 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.364430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:13.380833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.380778    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rjxn2"]
Jul 09 18:08:13.447556 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.447524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.447738 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.447579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e2efa792-8624-4922-a9ca-9617fdb18a02-crio-socket\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.447738 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.447654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.447738 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.447688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2efa792-8624-4922-a9ca-9617fdb18a02-data-volume\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.447738 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.447711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx4rj\" (UniqueName: \"kubernetes.io/projected/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-api-access-xx4rj\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548573 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.548539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e2efa792-8624-4922-a9ca-9617fdb18a02-crio-socket\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.548611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.548648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2efa792-8624-4922-a9ca-9617fdb18a02-data-volume\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.548671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx4rj\" (UniqueName: \"kubernetes.io/projected/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-api-access-xx4rj\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.548677    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e2efa792-8624-4922-a9ca-9617fdb18a02-crio-socket\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548967 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.548738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.548967 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:13.548771    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:13.548967 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:13.548848    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:14.048831951 +0000 UTC m=+256.395739850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:13.548967 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:13.548871    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:08:13.548967 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:13.548956    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:14.048935259 +0000 UTC m=+256.395843157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : configmap "kube-rbac-proxy" not found
Jul 09 18:08:13.549177 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.549046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2efa792-8624-4922-a9ca-9617fdb18a02-data-volume\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.574032 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.573996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx4rj\" (UniqueName: \"kubernetes.io/projected/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-api-access-xx4rj\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:13.577389 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:13.577361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:14.030349 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.030310    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:14.052543 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.052502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:14.052757 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.052579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:14.052757 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:14.052663    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:08:14.052757 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:14.052674    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:14.052757 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:14.052725    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:15.052705815 +0000 UTC m=+257.399613711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : configmap "kube-rbac-proxy" not found
Jul 09 18:08:14.052757 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:14.052740    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:15.052733677 +0000 UTC m=+257.399641570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:14.054181 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.054153    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"]
Jul 09 18:08:14.056700 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.056678    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"
Jul 09 18:08:14.062058 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.062034    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kw8xc\""
Jul 09 18:08:14.062186 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.062040    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 09 18:08:14.062186 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.062093    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 09 18:08:14.083190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.083162    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"]
Jul 09 18:08:14.128765 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.128734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:14.153952 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.153915    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5npph\" (UniqueName: \"kubernetes.io/projected/08575775-9374-4700-adbc-2a111c725eec-kube-api-access-5npph\") pod \"migrator-7f59b54f6c-k2rs2\" (UID: \"08575775-9374-4700-adbc-2a111c725eec\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"
Jul 09 18:08:14.229485 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.229447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:14.254758 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.254721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5npph\" (UniqueName: \"kubernetes.io/projected/08575775-9374-4700-adbc-2a111c725eec-kube-api-access-5npph\") pod \"migrator-7f59b54f6c-k2rs2\" (UID: \"08575775-9374-4700-adbc-2a111c725eec\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"
Jul 09 18:08:14.268176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.268146    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5npph\" (UniqueName: \"kubernetes.io/projected/08575775-9374-4700-adbc-2a111c725eec-kube-api-access-5npph\") pod \"migrator-7f59b54f6c-k2rs2\" (UID: \"08575775-9374-4700-adbc-2a111c725eec\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"
Jul 09 18:08:14.365903 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.365812    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"
Jul 09 18:08:14.498036 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.498002    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2"]
Jul 09 18:08:14.500650 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:14.500617    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08575775_9374_4700_adbc_2a111c725eec.slice/crio-5f124b5f9bfcf7af5aa87f157a3f33615a28632fe683a6ed8af69b5ccf8747b7 WatchSource:0}: Error finding container 5f124b5f9bfcf7af5aa87f157a3f33615a28632fe683a6ed8af69b5ccf8747b7: Status 404 returned error can't find the container with id 5f124b5f9bfcf7af5aa87f157a3f33615a28632fe683a6ed8af69b5ccf8747b7
Jul 09 18:08:14.946436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.946398    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-tqlcv"]
Jul 09 18:08:14.948920 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.948903    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:14.951208 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.951185    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 09 18:08:14.951313 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.951260    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vgmcd\""
Jul 09 18:08:14.952048 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.952030    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 09 18:08:14.952104 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.952067    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 09 18:08:14.952104 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.952092    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 09 18:08:14.957578 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.957554    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-tqlcv"]
Jul 09 18:08:14.983208 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:14.983181    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:15.053428 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.053393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:15.060101 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.060076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:15.060256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.060116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-signing-key\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.060256 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:15.060207    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:15.060256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.060207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:15.060256 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:15.060258    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:17.060243921 +0000 UTC m=+259.407151815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:15.060426 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:15.060272    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:08:15.060426 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:15.060365    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:17.060336346 +0000 UTC m=+259.407244243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : configmap "kube-rbac-proxy" not found
Jul 09 18:08:15.060491 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.060475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdsxq\" (UniqueName: \"kubernetes.io/projected/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-kube-api-access-rdsxq\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.060537 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.060522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-signing-cabundle\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.062088 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.062065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2" event={"ID":"08575775-9374-4700-adbc-2a111c725eec","Type":"ContainerStarted","Data":"5f124b5f9bfcf7af5aa87f157a3f33615a28632fe683a6ed8af69b5ccf8747b7"}
Jul 09 18:08:15.161467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.161429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdsxq\" (UniqueName: \"kubernetes.io/projected/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-kube-api-access-rdsxq\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.161663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.161480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-signing-cabundle\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.161663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.161527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-signing-key\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.162170 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.162147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-signing-cabundle\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.163958 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.163940    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-signing-key\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.170549 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.170517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdsxq\" (UniqueName: \"kubernetes.io/projected/a1ed2fe5-02f0-47ba-9010-c1161ff6e946-kube-api-access-rdsxq\") pod \"service-ca-79f68b9f4f-tqlcv\" (UID: \"a1ed2fe5-02f0-47ba-9010-c1161ff6e946\") " pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.258326 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.258235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv"
Jul 09 18:08:15.370807 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.370755    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rcdz4_cc25caa5-2d30-450f-958e-0c8b51386e5c/dns-node-resolver/0.log"
Jul 09 18:08:15.422693 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:15.422660    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-tqlcv"]
Jul 09 18:08:15.426850 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:15.426823    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ed2fe5_02f0_47ba_9010_c1161ff6e946.slice/crio-c4a1cea0a45794cd4952856c5fbdf1609bc730079bffd39166c87a098d29ec9c WatchSource:0}: Error finding container c4a1cea0a45794cd4952856c5fbdf1609bc730079bffd39166c87a098d29ec9c: Status 404 returned error can't find the container with id c4a1cea0a45794cd4952856c5fbdf1609bc730079bffd39166c87a098d29ec9c
Jul 09 18:08:16.067635 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.067545    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2" event={"ID":"08575775-9374-4700-adbc-2a111c725eec","Type":"ContainerStarted","Data":"47eebdee2527ec8d400f220d16ccf2bae5ac3c486bd1fde3e7cac9c006dc26b8"}
Jul 09 18:08:16.067635 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.067608    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2" event={"ID":"08575775-9374-4700-adbc-2a111c725eec","Type":"ContainerStarted","Data":"a15572b1b82c10d8927eff5c444af3051939f8092aeeec9d23de748ca1f49942"}
Jul 09 18:08:16.068839 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.068813    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv" event={"ID":"a1ed2fe5-02f0-47ba-9010-c1161ff6e946","Type":"ContainerStarted","Data":"c4a1cea0a45794cd4952856c5fbdf1609bc730079bffd39166c87a098d29ec9c"}
Jul 09 18:08:16.091176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.091093    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-k2rs2" podStartSLOduration=0.814718226 podStartE2EDuration="2.091072897s" totalImagesPullingTime="1.276354671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:16.090730864 +0000 UTC m=+258.437638780" watchObservedRunningTime="2026-07-09 18:08:16.091072897 +0000 UTC m=+258.437980818"
Jul 09 18:08:16.174126 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.174097    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9lw8l_dd6e4bba-e65a-4097-9a87-4d6ca3ac163a/node-ca/0.log"
Jul 09 18:08:16.322696 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.322609    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:16.373039 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.373001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:16.373501 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.373060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:16.373501 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:16.373182    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 18:08:16.373501 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:16.373181    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:16.373501 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:16.373252    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert podName:8bbaae80-06b4-43bf-92f1-def9f70426f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:24.373236686 +0000 UTC m=+266.720144580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hgdbm" (UID: "8bbaae80-06b4-43bf-92f1-def9f70426f4") : secret "networking-console-plugin-cert" not found
Jul 09 18:08:16.373501 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:16.373291    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls podName:e6c97669-d87d-4e2f-b35c-c7a39853020c nodeName:}" failed. No retries permitted until 2026-07-09 18:08:24.373272841 +0000 UTC m=+266.720180737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2qkks" (UID: "e6c97669-d87d-4e2f-b35c-c7a39853020c") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:16.474682 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:16.474645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:16.474888 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:16.474822    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:08:16.474966 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:16.474901    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls podName:f9296204-be27-4bae-8c31-3c88a9c08433 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:24.474880599 +0000 UTC m=+266.821788495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls") pod "cluster-samples-operator-65db858894-fzdlj" (UID: "f9296204-be27-4bae-8c31-3c88a9c08433") : secret "samples-operator-tls" not found
Jul 09 18:08:17.079823 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:17.079793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:17.080001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:17.079873    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:17.080001 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:17.079914    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:08:17.080001 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:17.079963    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:17.080001 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:17.079982    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:21.079965289 +0000 UTC m=+263.426873206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : configmap "kube-rbac-proxy" not found
Jul 09 18:08:17.080001 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:17.080001    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:21.079989923 +0000 UTC m=+263.426897817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:17.173758 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:17.173720    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hnmt_0236ff24-a897-4d0f-b0b0-23d3a7904f2b/serve-healthcheck-canary/0.log"
Jul 09 18:08:18.075448 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.075418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv" event={"ID":"a1ed2fe5-02f0-47ba-9010-c1161ff6e946","Type":"ContainerStarted","Data":"55fca3e274c148cf2e5aef8c58cce94092a7968d04f082f881260c7780b1215e"}
Jul 09 18:08:18.095436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.095375    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-tqlcv" podStartSLOduration=2.437936224 podStartE2EDuration="4.095360657s" totalImagesPullingTime="1.657424433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:18.093948141 +0000 UTC m=+260.440856058" watchObservedRunningTime="2026-07-09 18:08:18.095360657 +0000 UTC m=+260.442268572"
Jul 09 18:08:18.309484 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.309451    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:18.409499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.409406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:18.513164 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.513127    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:18.613770 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.613683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:18.854461 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.854425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:18.955053 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:18.955006    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:19.055176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:19.055144    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:19.567595 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:19.567559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:19.956766 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:19.956733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:20.742439 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:20.742407    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:20.800289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:20.800253    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:20.947803 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:20.947753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:21.115437 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:21.115347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:21.115437 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:21.115421    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:21.115666 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:21.115496    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:08:21.115666 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:21.115499    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:21.115666 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:21.115554    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:29.115537051 +0000 UTC m=+271.462444960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : configmap "kube-rbac-proxy" not found
Jul 09 18:08:21.115666 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:21.115568    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls podName:e2efa792-8624-4922-a9ca-9617fdb18a02 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:29.115562503 +0000 UTC m=+271.462470396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rjxn2" (UID: "e2efa792-8624-4922-a9ca-9617fdb18a02") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:08:21.753536 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:21.753508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:22.168122 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:22.168035    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:22.270367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:22.270329    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:22.367432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:22.367397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:22.566392 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:22.566352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:22.949305 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:22.949279    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:23.156557 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:23.156522    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:23.554075 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:23.554043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:24.347512 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.347477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:24.440663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.440619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:24.440663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.440662    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:24.440898 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:24.440775    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 18:08:24.440898 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:24.440846    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:24.440898 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:24.440854    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert podName:8bbaae80-06b4-43bf-92f1-def9f70426f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:40.440838881 +0000 UTC m=+282.787746775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hgdbm" (UID: "8bbaae80-06b4-43bf-92f1-def9f70426f4") : secret "networking-console-plugin-cert" not found
Jul 09 18:08:24.440898 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:24.440892    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls podName:e6c97669-d87d-4e2f-b35c-c7a39853020c nodeName:}" failed. No retries permitted until 2026-07-09 18:08:40.440876271 +0000 UTC m=+282.787784181 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2qkks" (UID: "e6c97669-d87d-4e2f-b35c-c7a39853020c") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:08:24.541435 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.541401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:24.543871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.543837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9296204-be27-4bae-8c31-3c88a9c08433-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fzdlj\" (UID: \"f9296204-be27-4bae-8c31-3c88a9c08433\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:24.557516 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.557489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:24.659680 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.659591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:24.672571 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.672543    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"
Jul 09 18:08:24.805581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.805502    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj"]
Jul 09 18:08:24.948278 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:24.948241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:25.099061 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:25.099028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj" event={"ID":"f9296204-be27-4bae-8c31-3c88a9c08433","Type":"ContainerStarted","Data":"7e57ce04494aee9441e383b8ae3907d30e32acfdda9fb0890df6fbbc646002b9"}
Jul 09 18:08:25.160830 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:25.160777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:25.554924 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:25.554887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:25.833249 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:25.833166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:26.175777 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:26.175697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:26.586099 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:26.586061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:26.954140 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:26.954101    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:27.152902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:27.152868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:28.049897 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.049819    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:28.103872 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.103831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:28.110747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.110720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj" event={"ID":"f9296204-be27-4bae-8c31-3c88a9c08433","Type":"ContainerStarted","Data":"5f5aa4c44b4c9f253ce8bf5479bb5a87aaa25afdc13453012219ce6c6e4c2d0e"}
Jul 09 18:08:28.110909 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.110777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj" event={"ID":"f9296204-be27-4bae-8c31-3c88a9c08433","Type":"ContainerStarted","Data":"71d7d74348737187e68a8491c1d45985f6f7861c6682e339c95da89c2b10068f"}
Jul 09 18:08:28.127014 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.126968    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fzdlj" podStartSLOduration=17.201396375 podStartE2EDuration="20.12695236s" totalImagesPullingTime="2.925555985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:28.125982836 +0000 UTC m=+270.472890771" watchObservedRunningTime="2026-07-09 18:08:28.12695236 +0000 UTC m=+270.473860272"
Jul 09 18:08:28.547382 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.547345    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:28.751317 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.751281    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:28.952846 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:28.952803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:29.183906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:29.183867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:29.184293 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:29.183981    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:29.184483 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:29.184456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e2efa792-8624-4922-a9ca-9617fdb18a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:29.186219 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:29.186196    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e2efa792-8624-4922-a9ca-9617fdb18a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjxn2\" (UID: \"e2efa792-8624-4922-a9ca-9617fdb18a02\") " pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:29.258151 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:29.258066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rjxn2"
Jul 09 18:08:29.387614 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:29.387581    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rjxn2"]
Jul 09 18:08:29.390299 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:29.390268    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2efa792_8624_4922_a9ca_9617fdb18a02.slice/crio-7bc9350fa3663c09e17094cdc8b225d456f3999907eb2372099ede417463cc6f WatchSource:0}: Error finding container 7bc9350fa3663c09e17094cdc8b225d456f3999907eb2372099ede417463cc6f: Status 404 returned error can't find the container with id 7bc9350fa3663c09e17094cdc8b225d456f3999907eb2372099ede417463cc6f
Jul 09 18:08:30.119407 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:30.119345    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjxn2" event={"ID":"e2efa792-8624-4922-a9ca-9617fdb18a02","Type":"ContainerStarted","Data":"7bc9350fa3663c09e17094cdc8b225d456f3999907eb2372099ede417463cc6f"}
Jul 09 18:08:32.380000 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:32.379962    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:32.449406 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:32.449356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:34.134694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:34.134606    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjxn2" event={"ID":"e2efa792-8624-4922-a9ca-9617fdb18a02","Type":"ContainerStarted","Data":"221772a88a48bd14af5d7eebe391f45e9cf7af441dc46e908215d797128417f7"}
Jul 09 18:08:34.951410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:34.951372    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:35.061161 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:35.061123    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:35.140453 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:35.140414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjxn2" event={"ID":"e2efa792-8624-4922-a9ca-9617fdb18a02","Type":"ContainerStarted","Data":"e5bb1fc0f8f8e57511125be9090c58a098b6b4d55d48ca14e9bc38c23ec52d04"}
Jul 09 18:08:35.858969 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:35.858935    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:35.951921 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:35.951880    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:37.149580 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:37.149485    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjxn2" event={"ID":"e2efa792-8624-4922-a9ca-9617fdb18a02","Type":"ContainerStarted","Data":"50987c77b3935a54f0ebad84d6dcac649c33b07c3b2f91e054a0671a6a5695f8"}
Jul 09 18:08:37.168959 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:37.168911    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rjxn2" podStartSLOduration=16.829210747 podStartE2EDuration="24.168894646s" totalImagesPullingTime="7.339683899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:08:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:37.167868253 +0000 UTC m=+279.514776169" watchObservedRunningTime="2026-07-09 18:08:37.168894646 +0000 UTC m=+279.515802539"
Jul 09 18:08:39.823331 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:39.823279    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lctck" podUID="cb03f821-623c-4a50-841d-8642fa93b2ef"
Jul 09 18:08:39.823331 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:39.823290    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"
Jul 09 18:08:40.159457 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.159374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:08:40.159596 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.159381    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lctck"
Jul 09 18:08:40.486322 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.486266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:40.486497 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.486334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:40.488717 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.488691    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bbaae80-06b4-43bf-92f1-def9f70426f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hgdbm\" (UID: \"8bbaae80-06b4-43bf-92f1-def9f70426f4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:40.488834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.488719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c97669-d87d-4e2f-b35c-c7a39853020c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2qkks\" (UID: \"e6c97669-d87d-4e2f-b35c-c7a39853020c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:40.755293 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.755175    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"
Jul 09 18:08:40.761473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.761451    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"
Jul 09 18:08:40.911561 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.911521    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks"]
Jul 09 18:08:40.914551 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:40.914522    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c97669_d87d_4e2f_b35c_c7a39853020c.slice/crio-d33e4504bc26f8b0cd4590572d44830bd641ab913024a5d0b3172424a0985355 WatchSource:0}: Error finding container d33e4504bc26f8b0cd4590572d44830bd641ab913024a5d0b3172424a0985355: Status 404 returned error can't find the container with id d33e4504bc26f8b0cd4590572d44830bd641ab913024a5d0b3172424a0985355
Jul 09 18:08:40.927269 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:40.927242    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-hgdbm"]
Jul 09 18:08:40.931174 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:40.931149    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bbaae80_06b4_43bf_92f1_def9f70426f4.slice/crio-8efa1c45e3678f474c81d439cbbcbed83c1c1d5c6703494675ae2aeccede7af2 WatchSource:0}: Error finding container 8efa1c45e3678f474c81d439cbbcbed83c1c1d5c6703494675ae2aeccede7af2: Status 404 returned error can't find the container with id 8efa1c45e3678f474c81d439cbbcbed83c1c1d5c6703494675ae2aeccede7af2
Jul 09 18:08:41.168284 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:41.168189    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm" event={"ID":"8bbaae80-06b4-43bf-92f1-def9f70426f4","Type":"ContainerStarted","Data":"8efa1c45e3678f474c81d439cbbcbed83c1c1d5c6703494675ae2aeccede7af2"}
Jul 09 18:08:41.169228 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:41.169204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks" event={"ID":"e6c97669-d87d-4e2f-b35c-c7a39853020c","Type":"ContainerStarted","Data":"d33e4504bc26f8b0cd4590572d44830bd641ab913024a5d0b3172424a0985355"}
Jul 09 18:08:43.134200 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.134107    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:43.178255 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.178223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm" event={"ID":"8bbaae80-06b4-43bf-92f1-def9f70426f4","Type":"ContainerStarted","Data":"0b5ca9c7b3ba2227fc11865bcda35ff4983d4177446228187c02e9e7bb8eed81"}
Jul 09 18:08:43.182719 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.182682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks" event={"ID":"e6c97669-d87d-4e2f-b35c-c7a39853020c","Type":"ContainerStarted","Data":"b7259328ef40aba90f27e48eba415a0cb8d974cfc4a45dcc754f2380ac4e84fc"}
Jul 09 18:08:43.205427 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.205381    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-hgdbm" podStartSLOduration=33.444537311 podStartE2EDuration="35.205367507s" totalImagesPullingTime="1.760830196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:43.203982297 +0000 UTC m=+285.550890234" watchObservedRunningTime="2026-07-09 18:08:43.205367507 +0000 UTC m=+285.552275413"
Jul 09 18:08:43.251753 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.251694    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2qkks" podStartSLOduration=33.470506272 podStartE2EDuration="35.251673842s" totalImagesPullingTime="1.78116757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:43.250260458 +0000 UTC m=+285.597168394" watchObservedRunningTime="2026-07-09 18:08:43.251673842 +0000 UTC m=+285.598581759"
Jul 09 18:08:43.409659 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.409561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:08:43.412043 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.412009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"image-registry-6f54f75b7c-874l5\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") " pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:08:43.435302 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.435264    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"]
Jul 09 18:08:43.439336 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.439319    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:43.452162 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.452142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 18:08:43.453698 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.453684    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7lwjq\""
Jul 09 18:08:43.466849 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.466831    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6mm8x\""
Jul 09 18:08:43.470521 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.470504    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:08:43.471047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.471027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:43.510272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.510234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:08:43.512718 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.512696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb03f821-623c-4a50-841d-8642fa93b2ef-metrics-tls\") pod \"dns-default-lctck\" (UID: \"cb03f821-623c-4a50-841d-8642fa93b2ef\") " pod="openshift-dns/dns-default-lctck"
Jul 09 18:08:43.538458 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.538404    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"]
Jul 09 18:08:43.610907 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.610872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d0b072b-ad0d-450a-a008-1f4094360d95-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xxt4p\" (UID: \"3d0b072b-ad0d-450a-a008-1f4094360d95\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:43.613042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.613010    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f54f75b7c-874l5"]
Jul 09 18:08:43.616452 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:43.616430    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac7d7af_a5b1_4814_8edb_8f3472e3beb8.slice/crio-2e57819847a6723265395fcb512a870bc63e8c00a7ec4d8925dcaae9bd4603f9 WatchSource:0}: Error finding container 2e57819847a6723265395fcb512a870bc63e8c00a7ec4d8925dcaae9bd4603f9: Status 404 returned error can't find the container with id 2e57819847a6723265395fcb512a870bc63e8c00a7ec4d8925dcaae9bd4603f9
Jul 09 18:08:43.622732 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.622710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:43.712103 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.712077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d0b072b-ad0d-450a-a008-1f4094360d95-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xxt4p\" (UID: \"3d0b072b-ad0d-450a-a008-1f4094360d95\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:43.714280 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.714257    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d0b072b-ad0d-450a-a008-1f4094360d95-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xxt4p\" (UID: \"3d0b072b-ad0d-450a-a008-1f4094360d95\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:43.734047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.734016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:43.748884 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.748860    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:43.764084 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.764059    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jd75b\""
Jul 09 18:08:43.770795 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.770755    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lctck"
Jul 09 18:08:43.843581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.843548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:43.908127 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.908096    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"]
Jul 09 18:08:43.911966 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:43.911940    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0b072b_ad0d_450a_a008_1f4094360d95.slice/crio-760911fdaa2ca57ab120fdf9ca5ed94751e43ed49e2d01f60031bad3f13aac4f WatchSource:0}: Error finding container 760911fdaa2ca57ab120fdf9ca5ed94751e43ed49e2d01f60031bad3f13aac4f: Status 404 returned error can't find the container with id 760911fdaa2ca57ab120fdf9ca5ed94751e43ed49e2d01f60031bad3f13aac4f
Jul 09 18:08:43.920923 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:43.920815    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lctck"]
Jul 09 18:08:43.924282 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:43.924254    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb03f821_623c_4a50_841d_8642fa93b2ef.slice/crio-b076b3c8944148877eeb55f098e8ed13389fdca9b0ca9d508b43b9321138193f WatchSource:0}: Error finding container b076b3c8944148877eeb55f098e8ed13389fdca9b0ca9d508b43b9321138193f: Status 404 returned error can't find the container with id b076b3c8944148877eeb55f098e8ed13389fdca9b0ca9d508b43b9321138193f
Jul 09 18:08:44.187422 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:44.187331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" event={"ID":"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8","Type":"ContainerStarted","Data":"2698d57c729646a5ce0e2788a41f4ed74b2e5e40dff82e5a7ce05ad3838ddefd"}
Jul 09 18:08:44.187422 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:44.187380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" event={"ID":"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8","Type":"ContainerStarted","Data":"2e57819847a6723265395fcb512a870bc63e8c00a7ec4d8925dcaae9bd4603f9"}
Jul 09 18:08:44.187422 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:44.187407    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:08:44.192575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:44.192550    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lctck" event={"ID":"cb03f821-623c-4a50-841d-8642fa93b2ef","Type":"ContainerStarted","Data":"b076b3c8944148877eeb55f098e8ed13389fdca9b0ca9d508b43b9321138193f"}
Jul 09 18:08:44.192717 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:44.192591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p" event={"ID":"3d0b072b-ad0d-450a-a008-1f4094360d95","Type":"ContainerStarted","Data":"760911fdaa2ca57ab120fdf9ca5ed94751e43ed49e2d01f60031bad3f13aac4f"}
Jul 09 18:08:44.208406 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:44.208367    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" podStartSLOduration=286.208354206 podStartE2EDuration="4m46.208354206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:44.208316589 +0000 UTC m=+286.555224506" watchObservedRunningTime="2026-07-09 18:08:44.208354206 +0000 UTC m=+286.555262123"
Jul 09 18:08:45.195327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:45.195270    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p" event={"ID":"3d0b072b-ad0d-450a-a008-1f4094360d95","Type":"ContainerStarted","Data":"111ae6d232b1dfecf11db6430191cfaf3348e7f36867e041f18854af91f080da"}
Jul 09 18:08:45.212651 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:45.212599    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p" podStartSLOduration=1.136816286 podStartE2EDuration="2.212584903s" totalImagesPullingTime="1.075768617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:45.212094031 +0000 UTC m=+287.559001972" watchObservedRunningTime="2026-07-09 18:08:45.212584903 +0000 UTC m=+287.559492843"
Jul 09 18:08:46.200004 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.199971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lctck" event={"ID":"cb03f821-623c-4a50-841d-8642fa93b2ef","Type":"ContainerStarted","Data":"4baea4baf558bfd1ed4850018507093d22b8a1ed040c8b938db482b0c083ff10"}
Jul 09 18:08:46.200394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.200027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lctck" event={"ID":"cb03f821-623c-4a50-841d-8642fa93b2ef","Type":"ContainerStarted","Data":"f5a8a04a2b95fe2d2ec78b1493db6276994d8c69590c4be10cdc1f5e37a32d68"}
Jul 09 18:08:46.200394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.200193    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:46.204871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.204850    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xxt4p"
Jul 09 18:08:46.218107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.218069    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lctck" podStartSLOduration=251.220576968 podStartE2EDuration="4m13.218055831s" totalImagesPullingTime="1.997478863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:46.216846409 +0000 UTC m=+288.563754343" watchObservedRunningTime="2026-07-09 18:08:46.218055831 +0000 UTC m=+288.564963746"
Jul 09 18:08:46.442507 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.442476    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:46.485604 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.485519    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-c59fq"]
Jul 09 18:08:46.509049 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.509022    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-c59fq"]
Jul 09 18:08:46.509201 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.509152    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.512456 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.512433    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 18:08:46.512456 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.512450    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 18:08:46.512631 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.512508    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8rnfs\""
Jul 09 18:08:46.512631 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.512525    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 18:08:46.536102 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.536069    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:46.636774 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.636709    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cckw\" (UniqueName: \"kubernetes.io/projected/5c0c272e-4bcc-4608-b033-7413930c4dda-kube-api-access-4cckw\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.637002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.636816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c0c272e-4bcc-4608-b033-7413930c4dda-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.637002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.636845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c0c272e-4bcc-4608-b033-7413930c4dda-metrics-client-ca\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.637002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.636872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c0c272e-4bcc-4608-b033-7413930c4dda-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.737747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.737650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cckw\" (UniqueName: \"kubernetes.io/projected/5c0c272e-4bcc-4608-b033-7413930c4dda-kube-api-access-4cckw\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.737747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.737698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c0c272e-4bcc-4608-b033-7413930c4dda-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.737747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.737725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c0c272e-4bcc-4608-b033-7413930c4dda-metrics-client-ca\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.738038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.737753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c0c272e-4bcc-4608-b033-7413930c4dda-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.738609 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.738565    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c0c272e-4bcc-4608-b033-7413930c4dda-metrics-client-ca\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.738890 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.738829    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:46.740385 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.740364    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c0c272e-4bcc-4608-b033-7413930c4dda-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.740471 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.740442    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c0c272e-4bcc-4608-b033-7413930c4dda-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.746556 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.746536    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cckw\" (UniqueName: \"kubernetes.io/projected/5c0c272e-4bcc-4608-b033-7413930c4dda-kube-api-access-4cckw\") pod \"prometheus-operator-64785b6675-c59fq\" (UID: \"5c0c272e-4bcc-4608-b033-7413930c4dda\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.819085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.819041    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq"
Jul 09 18:08:46.970085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:46.970056    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-c59fq"]
Jul 09 18:08:46.972719 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:46.972684    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0c272e_4bcc_4608_b033_7413930c4dda.slice/crio-6ba1ffafc64ca0a17dfed20d85f9abfbdb4c69b0dd1b5a131bf7147854ce94a7 WatchSource:0}: Error finding container 6ba1ffafc64ca0a17dfed20d85f9abfbdb4c69b0dd1b5a131bf7147854ce94a7: Status 404 returned error can't find the container with id 6ba1ffafc64ca0a17dfed20d85f9abfbdb4c69b0dd1b5a131bf7147854ce94a7
Jul 09 18:08:47.204846 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:47.204810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq" event={"ID":"5c0c272e-4bcc-4608-b033-7413930c4dda","Type":"ContainerStarted","Data":"6ba1ffafc64ca0a17dfed20d85f9abfbdb4c69b0dd1b5a131bf7147854ce94a7"}
Jul 09 18:08:47.205257 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:47.205238    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lctck"
Jul 09 18:08:48.443401 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:48.443369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:49.214963 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:49.214924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq" event={"ID":"5c0c272e-4bcc-4608-b033-7413930c4dda","Type":"ContainerStarted","Data":"99c2cddb9e13cc012a7d4c1bfd9f1740005f2338950126b8fac74cc377041874"}
Jul 09 18:08:49.215141 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:49.215019    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq" event={"ID":"5c0c272e-4bcc-4608-b033-7413930c4dda","Type":"ContainerStarted","Data":"96d20b8790b83951edd2363d128085ce621e52932885572c00ecf1e925bec171"}
Jul 09 18:08:49.233772 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:49.233723    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-c59fq" podStartSLOduration=2.066693963 podStartE2EDuration="3.233707567s" totalImagesPullingTime="1.167013604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:49.233091952 +0000 UTC m=+291.579999894" watchObservedRunningTime="2026-07-09 18:08:49.233707567 +0000 UTC m=+291.580615503"
Jul 09 18:08:50.615815 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.615765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:50.859300 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.859265    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:50.885351 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.885269    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"]
Jul 09 18:08:50.888095 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.888078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:50.890490 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.890465    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 18:08:50.890838 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.890820    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 18:08:50.890909 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.890844    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-742sd\""
Jul 09 18:08:50.904440 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.904417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"]
Jul 09 18:08:50.914464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.914435    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-47fhv"]
Jul 09 18:08:50.917030 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.917006    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:50.919543 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.919522    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 18:08:50.919648 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.919615    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xkn2w\""
Jul 09 18:08:50.919818 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.919803    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 18:08:50.919955 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.919937    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 18:08:50.968933 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:50.968892    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.062232 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.062184    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.069250 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069217    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-root\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069358 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/114cd342-a369-4587-bf22-786756baa888-node-exporter-textfile\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069358 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-node-exporter-wtmp\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069358 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/114cd342-a369-4587-bf22-786756baa888-metrics-client-ca\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069366    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/114cd342-a369-4587-bf22-786756baa888-node-exporter-accelerators-collector-config\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069397    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/114cd342-a369-4587-bf22-786756baa888-node-exporter-tls\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qv5z\" (UniqueName: \"kubernetes.io/projected/114cd342-a369-4587-bf22-786756baa888-kube-api-access-5qv5z\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-sys\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.069688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21121af4-c2c8-4f75-880a-b33e78df782b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.069688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069534    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrfrm\" (UniqueName: \"kubernetes.io/projected/21121af4-c2c8-4f75-880a-b33e78df782b-kube-api-access-jrfrm\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.069688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21121af4-c2c8-4f75-880a-b33e78df782b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.069688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21121af4-c2c8-4f75-880a-b33e78df782b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.069688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.069606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/114cd342-a369-4587-bf22-786756baa888-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.162941 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.162844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.170761 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170733    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrfrm\" (UniqueName: \"kubernetes.io/projected/21121af4-c2c8-4f75-880a-b33e78df782b-kube-api-access-jrfrm\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.170889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170765    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21121af4-c2c8-4f75-880a-b33e78df782b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.170889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21121af4-c2c8-4f75-880a-b33e78df782b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.170889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170840    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/114cd342-a369-4587-bf22-786756baa888-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-root\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/114cd342-a369-4587-bf22-786756baa888-node-exporter-textfile\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-node-exporter-wtmp\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170971    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-root\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.170985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/114cd342-a369-4587-bf22-786756baa888-metrics-client-ca\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/114cd342-a369-4587-bf22-786756baa888-node-exporter-accelerators-collector-config\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/114cd342-a369-4587-bf22-786756baa888-node-exporter-tls\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171102    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qv5z\" (UniqueName: \"kubernetes.io/projected/114cd342-a369-4587-bf22-786756baa888-kube-api-access-5qv5z\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171143    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-node-exporter-wtmp\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-sys\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171288 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21121af4-c2c8-4f75-880a-b33e78df782b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.171575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171323    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/114cd342-a369-4587-bf22-786756baa888-node-exporter-textfile\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/114cd342-a369-4587-bf22-786756baa888-sys\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171755 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171735    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/114cd342-a369-4587-bf22-786756baa888-metrics-client-ca\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.171895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.171875    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/114cd342-a369-4587-bf22-786756baa888-node-exporter-accelerators-collector-config\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.172115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.172093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21121af4-c2c8-4f75-880a-b33e78df782b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.173823 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.173767    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/114cd342-a369-4587-bf22-786756baa888-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.173916 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.173872    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/114cd342-a369-4587-bf22-786756baa888-node-exporter-tls\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.174089 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.174073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21121af4-c2c8-4f75-880a-b33e78df782b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.174316 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.174296    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21121af4-c2c8-4f75-880a-b33e78df782b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.180232 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.180206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qv5z\" (UniqueName: \"kubernetes.io/projected/114cd342-a369-4587-bf22-786756baa888-kube-api-access-5qv5z\") pod \"node-exporter-47fhv\" (UID: \"114cd342-a369-4587-bf22-786756baa888\") " pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.180770 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.180740    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrfrm\" (UniqueName: \"kubernetes.io/projected/21121af4-c2c8-4f75-880a-b33e78df782b-kube-api-access-jrfrm\") pod \"openshift-state-metrics-84bd68b896-s68lq\" (UID: \"21121af4-c2c8-4f75-880a-b33e78df782b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.199205 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.199142    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"
Jul 09 18:08:51.229111 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.229071    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-47fhv"
Jul 09 18:08:51.262748 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.262720    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.336652 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.336628    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq"]
Jul 09 18:08:51.339143 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:51.339111    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21121af4_c2c8_4f75_880a_b33e78df782b.slice/crio-6cf1d507083df3398784e65cfd65b462503778ef043f4e176312cd2b69959c8d WatchSource:0}: Error finding container 6cf1d507083df3398784e65cfd65b462503778ef043f4e176312cd2b69959c8d: Status 404 returned error can't find the container with id 6cf1d507083df3398784e65cfd65b462503778ef043f4e176312cd2b69959c8d
Jul 09 18:08:51.339679 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.339647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.403071 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.403041    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.852654 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.852601    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.945747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.945690    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:51.949325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.949299    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:08:51.952778 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.952758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.955458 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.955407    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 18:08:51.955458 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.955444    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 18:08:51.955804 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.955673    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 18:08:51.955974 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.955926    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 18:08:51.956121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.955956    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 18:08:51.956121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.956114    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 18:08:51.956280 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.956261    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 18:08:51.956280 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.955975    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 18:08:51.956613 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.956596    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dwlw2\""
Jul 09 18:08:51.959318 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.959041    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 18:08:51.978396 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.978366    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980367    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980408    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k687q\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-kube-api-access-k687q\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980559    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-config-volume\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.980617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.981175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980658    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.981175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980700    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.981175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.981175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-config-out\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:51.981175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:51.980879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-web-config\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.081573 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081537    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-config-volume\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.081759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081589    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.081759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.081759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.081759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081693    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.081759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.082065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-config-out\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.082065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-web-config\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.082065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081840    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.082065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.082065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.081897    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k687q\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-kube-api-access-k687q\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.082327 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:52.082216    2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 09 18:08:52.083647 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.082373    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.083647 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.083230    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.083647 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:52.083540    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle podName:c91103d5-115a-4385-97a6-ceeae55d2483 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:52.583515832 +0000 UTC m=+294.930423740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483") : configmap references non-existent config key: ca-bundle.crt
Jul 09 18:08:52.083906 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:52.083808    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls podName:c91103d5-115a-4385-97a6-ceeae55d2483 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:52.58377408 +0000 UTC m=+294.930681978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483") : secret "alertmanager-main-tls" not found
Jul 09 18:08:52.083906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.083866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.083906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.083899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.084916 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.084819    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-config-volume\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.085415 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.085389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.086660 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.086634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.087570 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.087529    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.087570 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.087548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-config-out\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.088385 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.088335    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-web-config\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.088917 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.088869    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.090433 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.090411    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.090851 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.090830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k687q\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-kube-api-access-k687q\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.228859 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.228820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq" event={"ID":"21121af4-c2c8-4f75-880a-b33e78df782b","Type":"ContainerStarted","Data":"46dc91eb389307ecc01bcb0a539cbc9c3a3bff9b0e8da44e9b5d21350db483e5"}
Jul 09 18:08:52.229056 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.228896    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq" event={"ID":"21121af4-c2c8-4f75-880a-b33e78df782b","Type":"ContainerStarted","Data":"dc608277fec21d6016c637606a250a59c96fe4ba877036f94114f4cf60713923"}
Jul 09 18:08:52.229056 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.228938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq" event={"ID":"21121af4-c2c8-4f75-880a-b33e78df782b","Type":"ContainerStarted","Data":"6cf1d507083df3398784e65cfd65b462503778ef043f4e176312cd2b69959c8d"}
Jul 09 18:08:52.230218 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.230150    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-47fhv" event={"ID":"114cd342-a369-4587-bf22-786756baa888","Type":"ContainerStarted","Data":"65b8b28640a661691d30ae74319388d3a885fc868c3d93876cc92b98af8284d7"}
Jul 09 18:08:52.365355 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.365303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:52.588327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.588290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.588327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.588331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.589221 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.589190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.590839 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.590810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:52.864417 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:52.864318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:08:53.006383 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.006347    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:08:53.009508 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:53.009471    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91103d5_115a_4385_97a6_ceeae55d2483.slice/crio-228cccbe08a7299ce61e49b996f0351862a3135490de355b04260422dd86ed0a WatchSource:0}: Error finding container 228cccbe08a7299ce61e49b996f0351862a3135490de355b04260422dd86ed0a: Status 404 returned error can't find the container with id 228cccbe08a7299ce61e49b996f0351862a3135490de355b04260422dd86ed0a
Jul 09 18:08:53.190011 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:08:53.189980    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nm5xr" podUID="bcfc3b54-df61-4210-b823-da5675e01562"
Jul 09 18:08:53.235287 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.235252    2567 generic.go:356] "Generic (PLEG): container finished" podID="114cd342-a369-4587-bf22-786756baa888" containerID="38f6efce28fec650839b4bfda778d7588b320e3e0388b9c9f17dd5b6e95bce8d" exitCode=0
Jul 09 18:08:53.235469 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.235341    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-47fhv" event={"ID":"114cd342-a369-4587-bf22-786756baa888","Type":"ContainerDied","Data":"38f6efce28fec650839b4bfda778d7588b320e3e0388b9c9f17dd5b6e95bce8d"}
Jul 09 18:08:53.236614 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.236581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"228cccbe08a7299ce61e49b996f0351862a3135490de355b04260422dd86ed0a"}
Jul 09 18:08:53.238737 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.238715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq" event={"ID":"21121af4-c2c8-4f75-880a-b33e78df782b","Type":"ContainerStarted","Data":"c98d37debc383dbe1bc69ef50d2928bd12fcf4a33348669ccee4b133b68267cd"}
Jul 09 18:08:53.279170 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.279123    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s68lq" podStartSLOduration=2.245629893 podStartE2EDuration="3.279106319s" totalImagesPullingTime="1.033476426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:53.278303465 +0000 UTC m=+295.625211406" watchObservedRunningTime="2026-07-09 18:08:53.279106319 +0000 UTC m=+295.626014234"
Jul 09 18:08:53.812747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.812658    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:53.891557 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.891529    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"]
Jul 09 18:08:53.895757 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.895731    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.898992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.898967    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 18:08:53.898992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.898969    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 18:08:53.899227 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.899211    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a36eiimrmt4fb\""
Jul 09 18:08:53.899299 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.899281    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 18:08:53.899352 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.899299    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 18:08:53.899401 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.899294    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 18:08:53.899451 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.899291    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mbhgf\""
Jul 09 18:08:53.900915 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.900893    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kfkx\" (UniqueName: \"kubernetes.io/projected/029ed26f-317b-4f04-88d0-3a72273a8c87-kube-api-access-7kfkx\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.900929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.900971    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.901000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-tls\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901163 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.901017    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901163 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.901098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901163 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.901150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029ed26f-317b-4f04-88d0-3a72273a8c87-metrics-client-ca\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.901310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.901209    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-grpc-tls\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:53.935265 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:53.935237    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"]
Jul 09 18:08:54.002260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002212    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-tls\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002314    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002414    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029ed26f-317b-4f04-88d0-3a72273a8c87-metrics-client-ca\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002649 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-grpc-tls\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002649 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kfkx\" (UniqueName: \"kubernetes.io/projected/029ed26f-317b-4f04-88d0-3a72273a8c87-kube-api-access-7kfkx\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.002649 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.002522    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.003470 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.003360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029ed26f-317b-4f04-88d0-3a72273a8c87-metrics-client-ca\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.005351 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.005326    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-tls\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.005485 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.005403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.005772 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.005752    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.006183 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.006168    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.006547 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.006532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.006818 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.006773    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/029ed26f-317b-4f04-88d0-3a72273a8c87-secret-grpc-tls\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.014412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.014386    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kfkx\" (UniqueName: \"kubernetes.io/projected/029ed26f-317b-4f04-88d0-3a72273a8c87-kube-api-access-7kfkx\") pod \"thanos-querier-5b79df75c8-lwp5k\" (UID: \"029ed26f-317b-4f04-88d0-3a72273a8c87\") " pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.231289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.231256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:54.246303 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.246237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-47fhv" event={"ID":"114cd342-a369-4587-bf22-786756baa888","Type":"ContainerStarted","Data":"ae8c0d3b9c9322bd3604c5d4658e2791583937c95bcedd34ea210456506e45a7"}
Jul 09 18:08:54.246447 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.246344    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-47fhv" event={"ID":"114cd342-a369-4587-bf22-786756baa888","Type":"ContainerStarted","Data":"bad7c80453644b8b821846268a3c8a8bf909d1aca90f54599dfeeb4941dbd041"}
Jul 09 18:08:54.248464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.248358    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f" exitCode=0
Jul 09 18:08:54.248464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.248448    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f"}
Jul 09 18:08:54.264985 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.264930    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-47fhv" podStartSLOduration=2.987829053 podStartE2EDuration="4.264902188s" totalImagesPullingTime="1.206714667s" totalInitContainerRuntime="70.358468ms" isStatefulPod=true podCreationTimestamp="2026-07-09 18:08:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:54.263431267 +0000 UTC m=+296.610339205" watchObservedRunningTime="2026-07-09 18:08:54.264902188 +0000 UTC m=+296.611810105"
Jul 09 18:08:54.371407 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:54.371372    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"]
Jul 09 18:08:54.374334 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:54.374307    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029ed26f_317b_4f04_88d0_3a72273a8c87.slice/crio-0b478b1b5d1fd8e1fd8302d9162fe80e643c8c14c2fe791227b6f206d1cc13b2 WatchSource:0}: Error finding container 0b478b1b5d1fd8e1fd8302d9162fe80e643c8c14c2fe791227b6f206d1cc13b2: Status 404 returned error can't find the container with id 0b478b1b5d1fd8e1fd8302d9162fe80e643c8c14c2fe791227b6f206d1cc13b2
Jul 09 18:08:55.221917 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.221883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:55.261902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.261856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"0b478b1b5d1fd8e1fd8302d9162fe80e643c8c14c2fe791227b6f206d1cc13b2"}
Jul 09 18:08:55.322958 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.322908    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-58c7786f4b-k57cq"]
Jul 09 18:08:55.327103 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.327078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.330628 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.330555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2pukk5jcj6rjd\""
Jul 09 18:08:55.330628 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.330609    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6zcfr\""
Jul 09 18:08:55.330628 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.330629    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 18:08:55.330871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.330652    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 18:08:55.330871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.330669    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 18:08:55.330993 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.330974    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 18:08:55.338480 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.337779    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58c7786f4b-k57cq"]
Jul 09 18:08:55.417698 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.417642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/203d4f89-7850-46c7-952d-c6c8c0730ec0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.417698 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.417693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5nlt\" (UniqueName: \"kubernetes.io/projected/203d4f89-7850-46c7-952d-c6c8c0730ec0-kube-api-access-p5nlt\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.417991 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.417759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/203d4f89-7850-46c7-952d-c6c8c0730ec0-metrics-server-audit-profiles\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.417991 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.417862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/203d4f89-7850-46c7-952d-c6c8c0730ec0-audit-log\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.417991 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.417898    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-secret-metrics-server-client-certs\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.417991 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.417947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-client-ca-bundle\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.418184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.418020    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-secret-metrics-server-tls\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519254 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-client-ca-bundle\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519254 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-secret-metrics-server-tls\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519495 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/203d4f89-7850-46c7-952d-c6c8c0730ec0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519495 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5nlt\" (UniqueName: \"kubernetes.io/projected/203d4f89-7850-46c7-952d-c6c8c0730ec0-kube-api-access-p5nlt\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519495 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/203d4f89-7850-46c7-952d-c6c8c0730ec0-metrics-server-audit-profiles\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519495 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/203d4f89-7850-46c7-952d-c6c8c0730ec0-audit-log\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519495 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519419    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-secret-metrics-server-client-certs\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.519994 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.519965    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/203d4f89-7850-46c7-952d-c6c8c0730ec0-audit-log\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.520165 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.520130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/203d4f89-7850-46c7-952d-c6c8c0730ec0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.520462 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.520438    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/203d4f89-7850-46c7-952d-c6c8c0730ec0-metrics-server-audit-profiles\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.522308 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.522272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-secret-metrics-server-client-certs\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.522413 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.522360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-secret-metrics-server-tls\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.522492 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.522469    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/203d4f89-7850-46c7-952d-c6c8c0730ec0-client-ca-bundle\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.528616 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.528591    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5nlt\" (UniqueName: \"kubernetes.io/projected/203d4f89-7850-46c7-952d-c6c8c0730ec0-kube-api-access-p5nlt\") pod \"metrics-server-58c7786f4b-k57cq\" (UID: \"203d4f89-7850-46c7-952d-c6c8c0730ec0\") " pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:55.613290 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.613251    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:55.641115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:55.641085    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:08:56.137498 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.137468    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"]
Jul 09 18:08:56.141312 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.141289    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.144059 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.144032    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 18:08:56.144227 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.144131    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 18:08:56.144338 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.144264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 18:08:56.144338 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.144294    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 18:08:56.144468 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.144449    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 18:08:56.144468 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.144456    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vsvmk\""
Jul 09 18:08:56.150367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.150101    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 18:08:56.170118 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.170087    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"]
Jul 09 18:08:56.202731 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.202676    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:56.225171 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225140    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-secret-telemeter-client\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225180    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-serving-certs-ca-bundle\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-metrics-client-ca\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-telemeter-client-tls\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-federate-client-tls\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225401    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhrx\" (UniqueName: \"kubernetes.io/projected/ffb23c53-46ab-47e3-b760-f5bd4e30019f-kube-api-access-5qhrx\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.225523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.225426    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326620 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-metrics-client-ca\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326699 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326733 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-telemeter-client-tls\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326773 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-federate-client-tls\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326848 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326766    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhrx\" (UniqueName: \"kubernetes.io/projected/ffb23c53-46ab-47e3-b760-f5bd4e30019f-kube-api-access-5qhrx\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326848 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326911 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-secret-telemeter-client\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.326941 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.326917    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-serving-certs-ca-bundle\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.328716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.327421    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-metrics-client-ca\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.328716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.327591    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-serving-certs-ca-bundle\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.328716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.328493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb23c53-46ab-47e3-b760-f5bd4e30019f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.330550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.330486    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-telemeter-client-tls\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.330550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.330533    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-secret-telemeter-client\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.332943 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.331946    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-federate-client-tls\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.332943 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.332118    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffb23c53-46ab-47e3-b760-f5bd4e30019f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.338685 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.338627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhrx\" (UniqueName: \"kubernetes.io/projected/ffb23c53-46ab-47e3-b760-f5bd4e30019f-kube-api-access-5qhrx\") pod \"telemeter-client-66dbd5c75-zxkf9\" (UID: \"ffb23c53-46ab-47e3-b760-f5bd4e30019f\") " pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.446336 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.446316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58c7786f4b-k57cq"]
Jul 09 18:08:56.450597 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:56.450546    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203d4f89_7850_46c7_952d_c6c8c0730ec0.slice/crio-572f4382b8591f05a9413f570bd38d3fc731aba18e01cedb93302055b474c59f WatchSource:0}: Error finding container 572f4382b8591f05a9413f570bd38d3fc731aba18e01cedb93302055b474c59f: Status 404 returned error can't find the container with id 572f4382b8591f05a9413f570bd38d3fc731aba18e01cedb93302055b474c59f
Jul 09 18:08:56.452959 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.452883    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"
Jul 09 18:08:56.651339 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:56.651316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66dbd5c75-zxkf9"]
Jul 09 18:08:56.654425 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:08:56.654393    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb23c53_46ab_47e3_b760_f5bd4e30019f.slice/crio-1115d94517adce953eb5732e49d4e4c50ae78695dc5615049737ef38170cb603 WatchSource:0}: Error finding container 1115d94517adce953eb5732e49d4e4c50ae78695dc5615049737ef38170cb603: Status 404 returned error can't find the container with id 1115d94517adce953eb5732e49d4e4c50ae78695dc5615049737ef38170cb603
Jul 09 18:08:57.114751 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.114712    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:57.213283 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.212445    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lctck"
Jul 09 18:08:57.277079 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.277041    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"e192d48a9bc56402ee78e9364471264ae918953b21ed0e92f27f7d071a25c6c3"}
Jul 09 18:08:57.277741 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.277707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"1fca8194897800ced59541575243329e4dcd57ed177ac71e653696057bb144b4"}
Jul 09 18:08:57.278466 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.278443    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"31b7520f117f290ff262bd25a0ce276c5a709d066966b758b55eaedefe590a92"}
Jul 09 18:08:57.281372 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.281319    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq" event={"ID":"203d4f89-7850-46c7-952d-c6c8c0730ec0","Type":"ContainerStarted","Data":"572f4382b8591f05a9413f570bd38d3fc731aba18e01cedb93302055b474c59f"}
Jul 09 18:08:57.289225 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.289199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9" event={"ID":"ffb23c53-46ab-47e3-b760-f5bd4e30019f","Type":"ContainerStarted","Data":"1115d94517adce953eb5732e49d4e4c50ae78695dc5615049737ef38170cb603"}
Jul 09 18:08:57.294695 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.294486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c"}
Jul 09 18:08:57.294695 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.294567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761"}
Jul 09 18:08:57.294695 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.294612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd"}
Jul 09 18:08:57.295095 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.294990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1"}
Jul 09 18:08:57.295095 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:57.295049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0"}
Jul 09 18:08:58.303416 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.303240    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerStarted","Data":"b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b"}
Jul 09 18:08:58.328325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.306465    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"ebf97288e7a353435d5ede5a0b3bf1dd21eb5db9a64c879ae71372f175383709"}
Jul 09 18:08:58.328325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.306524    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"70eaa17d58e41f76ddebe4ff13c3f5a1131156879b38e83738edcf367341cd18"}
Jul 09 18:08:58.328325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.313535    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f54f75b7c-874l5"]
Jul 09 18:08:58.328325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.319174    2567 patch_prober.go:28] interesting pod/image-registry-6f54f75b7c-874l5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 18:08:58.328325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.319227    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 18:08:58.340424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.340364    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.578738795 podStartE2EDuration="7.340340732s" totalImagesPullingTime="3.747523346s" totalInitContainerRuntime="14.078591ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:58.338728425 +0000 UTC m=+300.685636342" watchObservedRunningTime="2026-07-09 18:08:58.340340732 +0000 UTC m=+300.687248671"
Jul 09 18:08:58.410284 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.410198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:58.509873 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.509823    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:58.721044 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.721021    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:08:58.721985 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.721965    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:08:58.727460 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:58.727442    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 18:08:59.313968 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.313935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" event={"ID":"029ed26f-317b-4f04-88d0-3a72273a8c87","Type":"ContainerStarted","Data":"54e5dda107df684da28bfdfc700aa7ab4b27ef95f62d5048128ec54f977f9293"}
Jul 09 18:08:59.314428 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.314066    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:08:59.315404 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.315382    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq" event={"ID":"203d4f89-7850-46c7-952d-c6c8c0730ec0","Type":"ContainerStarted","Data":"5d57d2bbe9f325bbcfab7881eb19787ff2ce63deb5aa16f8bc8ee1bb5a9b23f5"}
Jul 09 18:08:59.317310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.317284    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9" event={"ID":"ffb23c53-46ab-47e3-b760-f5bd4e30019f","Type":"ContainerStarted","Data":"3e0e5ebdb1808e8fa217006b8aa98473c278ac9f88098bc8a49f98443ab930e7"}
Jul 09 18:08:59.317412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.317333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9" event={"ID":"ffb23c53-46ab-47e3-b760-f5bd4e30019f","Type":"ContainerStarted","Data":"13fffa88f477776086f9e999fd6a592adfc9136ff988cb0c15c063f7d903287f"}
Jul 09 18:08:59.317412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.317358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9" event={"ID":"ffb23c53-46ab-47e3-b760-f5bd4e30019f","Type":"ContainerStarted","Data":"d66067bb662cd4e485478b929fffadccebd96557e6d4d0f8c57595fd5510c5fd"}
Jul 09 18:08:59.339135 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.339079    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k" podStartSLOduration=3.417545501 podStartE2EDuration="6.339059124s" totalImagesPullingTime="2.921513623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:59.337589447 +0000 UTC m=+301.684497368" watchObservedRunningTime="2026-07-09 18:08:59.339059124 +0000 UTC m=+301.685967043"
Jul 09 18:08:59.357500 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.357450    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq" podStartSLOduration=2.141427795 podStartE2EDuration="4.357433092s" totalImagesPullingTime="2.216005297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:59.35696706 +0000 UTC m=+301.703874998" watchObservedRunningTime="2026-07-09 18:08:59.357433092 +0000 UTC m=+301.704341008"
Jul 09 18:08:59.381592 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:08:59.381545    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-66dbd5c75-zxkf9" podStartSLOduration=1.365125184 podStartE2EDuration="3.381528668s" totalImagesPullingTime="2.016403484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:59.380618331 +0000 UTC m=+301.727526272" watchObservedRunningTime="2026-07-09 18:08:59.381528668 +0000 UTC m=+301.728436584"
Jul 09 18:09:02.382115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:02.382072    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:02.480978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:02.480928    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:04.189470 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:04.189439    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:09:05.327765 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:05.327740    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b79df75c8-lwp5k"
Jul 09 18:09:06.392625 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:06.392592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:06.493797 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:06.493751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:08.318859 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.318831    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:09:08.384814 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.384764    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-fk6jw"]
Jul 09 18:09:08.390387 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.390354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:08.400535 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.400500    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fk6jw"]
Jul 09 18:09:08.400669 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.400620    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:08.403151 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.403121    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 18:09:08.403282 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.403121    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2f66t\""
Jul 09 18:09:08.403282 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.403161    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 18:09:08.490958 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.490923    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:08.549098 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.549060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:08.552171 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.552148    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-757pn\" (UniqueName: \"kubernetes.io/projected/e30217fd-44c2-41e6-b154-6e9d2e35ce6f-kube-api-access-757pn\") pod \"downloads-68c9cbb497-fk6jw\" (UID: \"e30217fd-44c2-41e6-b154-6e9d2e35ce6f\") " pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:08.653671 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.653578    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-757pn\" (UniqueName: \"kubernetes.io/projected/e30217fd-44c2-41e6-b154-6e9d2e35ce6f-kube-api-access-757pn\") pod \"downloads-68c9cbb497-fk6jw\" (UID: \"e30217fd-44c2-41e6-b154-6e9d2e35ce6f\") " pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:08.662537 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.662501    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-757pn\" (UniqueName: \"kubernetes.io/projected/e30217fd-44c2-41e6-b154-6e9d2e35ce6f-kube-api-access-757pn\") pod \"downloads-68c9cbb497-fk6jw\" (UID: \"e30217fd-44c2-41e6-b154-6e9d2e35ce6f\") " pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:08.678275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.678246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:08.710876 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.710841    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:08.844392 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.844361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fk6jw"]
Jul 09 18:09:08.847701 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:09:08.847672    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30217fd_44c2_41e6_b154_6e9d2e35ce6f.slice/crio-8c1506e67523cb62e3a33781e2eaebc582c577dbecbe74b1be51c720bb9379e4 WatchSource:0}: Error finding container 8c1506e67523cb62e3a33781e2eaebc582c577dbecbe74b1be51c720bb9379e4: Status 404 returned error can't find the container with id 8c1506e67523cb62e3a33781e2eaebc582c577dbecbe74b1be51c720bb9379e4
Jul 09 18:09:08.849972 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:08.849953    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:09:09.155252 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.155213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:09.212173 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.212139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:09.360342 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.360308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fk6jw" event={"ID":"e30217fd-44c2-41e6-b154-6e9d2e35ce6f","Type":"ContainerStarted","Data":"8c1506e67523cb62e3a33781e2eaebc582c577dbecbe74b1be51c720bb9379e4"}
Jul 09 18:09:09.441870 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.441835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:09.494480 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.494437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:09.642702 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.642662    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:09.857513 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:09.857406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:10.055927 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:10.055889    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:10.624421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:10.624393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:11.200772 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:11.200740    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:11.253561 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:11.253514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:11.680300 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:11.680203    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:11.737443 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:11.737411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:12.116159 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:12.116087    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:12.182074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:12.182037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:14.704002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:14.703936    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:14.795757 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:14.795708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:15.641459 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:15.641417    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:09:15.641622 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:15.641483    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:09:17.373844 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.373802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:17.474779 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.474734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:17.938884 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.938854    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b478b456-92c5q"]
Jul 09 18:09:17.949072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.949042    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:17.953172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.952552    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 18:09:17.953172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.952668    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 18:09:17.953172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.952863    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 18:09:17.953172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.952922    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 18:09:17.953172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.952964    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 18:09:17.953172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.953092    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7bkfr\""
Jul 09 18:09:17.953970 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:17.953938    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b478b456-92c5q"]
Jul 09 18:09:18.048238 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.048201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-console-config\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.048420 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.048276    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7xxh\" (UniqueName: \"kubernetes.io/projected/5b744141-9725-4d36-a8d7-551dc52d6b07-kube-api-access-j7xxh\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.048420 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.048339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-serving-cert\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.048420 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.048365    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-oauth-config\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.048551 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.048430    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-service-ca\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.048551 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.048454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-oauth-serving-cert\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.149779 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.149738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-console-config\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.149978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.149834    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7xxh\" (UniqueName: \"kubernetes.io/projected/5b744141-9725-4d36-a8d7-551dc52d6b07-kube-api-access-j7xxh\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.149978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.149896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-serving-cert\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.149978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.149921    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-oauth-config\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.150159 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.149985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-service-ca\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.150214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.150167    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-oauth-serving-cert\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.150839 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.150811    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-service-ca\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.150839 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.150824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-console-config\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.151105 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.150863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-oauth-serving-cert\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.153046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.152997    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-oauth-config\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.153144 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.153093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-serving-cert\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.157881 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.157856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7xxh\" (UniqueName: \"kubernetes.io/projected/5b744141-9725-4d36-a8d7-551dc52d6b07-kube-api-access-j7xxh\") pod \"console-77b478b456-92c5q\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") " pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.262970 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.262930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:18.413738 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:18.413704    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b478b456-92c5q"]
Jul 09 18:09:18.416531 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:09:18.416500    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b744141_9725_4d36_a8d7_551dc52d6b07.slice/crio-3dd2947c725d5a1f02df0ab5f78106337fa73aad12a6656b3145f981d74c27ce WatchSource:0}: Error finding container 3dd2947c725d5a1f02df0ab5f78106337fa73aad12a6656b3145f981d74c27ce: Status 404 returned error can't find the container with id 3dd2947c725d5a1f02df0ab5f78106337fa73aad12a6656b3145f981d74c27ce
Jul 09 18:09:19.411414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:19.411375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b478b456-92c5q" event={"ID":"5b744141-9725-4d36-a8d7-551dc52d6b07","Type":"ContainerStarted","Data":"3dd2947c725d5a1f02df0ab5f78106337fa73aad12a6656b3145f981d74c27ce"}
Jul 09 18:09:20.207709 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:20.207675    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:20.277257 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:20.277222    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:23.336883 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:23.336845    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" containerName="registry" containerID="cri-o://2698d57c729646a5ce0e2788a41f4ed74b2e5e40dff82e5a7ce05ad3838ddefd" gracePeriod=30
Jul 09 18:09:23.481659 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:23.481624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:26.897422 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:26.897385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:26.992184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:26.992148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:27.453339 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.453296    2567 generic.go:356] "Generic (PLEG): container finished" podID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" containerID="2698d57c729646a5ce0e2788a41f4ed74b2e5e40dff82e5a7ce05ad3838ddefd" exitCode=0
Jul 09 18:09:27.453505 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.453351    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" event={"ID":"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8","Type":"ContainerDied","Data":"2698d57c729646a5ce0e2788a41f4ed74b2e5e40dff82e5a7ce05ad3838ddefd"}
Jul 09 18:09:27.858010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.857984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:09:27.948082 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948060    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-certificates\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948106    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948139    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpzs4\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-kube-api-access-qpzs4\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948170    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-ca-trust-extracted\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948190    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-bound-sa-token\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948218    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-installation-pull-secrets\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948255    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-image-registry-private-configuration\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948343    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-trusted-ca\") pod \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\" (UID: \"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8\") "
Jul 09 18:09:27.948473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948448    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:09:27.948870 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948799    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:09:27.948870 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.948824    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-certificates\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:27.951546 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.951519    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:09:27.951667 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.951645    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:09:27.951738 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.951721    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:09:27.951816 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.951746    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-kube-api-access-qpzs4" (OuterVolumeSpecName: "kube-api-access-qpzs4") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "kube-api-access-qpzs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:09:27.952257 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.952235    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:09:27.957655 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:27.957629    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" (UID: "0ac7d7af-a5b1-4814-8edb-8f3472e3beb8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:09:28.050176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050140    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-trusted-ca\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.050176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050166    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-registry-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.050176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050177    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpzs4\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-kube-api-access-qpzs4\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.050394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050187    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-ca-trust-extracted\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.050394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050198    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-bound-sa-token\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.050394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050207    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-installation-pull-secrets\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.050394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.050216    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8-image-registry-private-configuration\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:09:28.460168 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.460092    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5"
Jul 09 18:09:28.460510 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.460092    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f54f75b7c-874l5" event={"ID":"0ac7d7af-a5b1-4814-8edb-8f3472e3beb8","Type":"ContainerDied","Data":"2e57819847a6723265395fcb512a870bc63e8c00a7ec4d8925dcaae9bd4603f9"}
Jul 09 18:09:28.460720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.460699    2567 scope.go:117] "RemoveContainer" containerID="2698d57c729646a5ce0e2788a41f4ed74b2e5e40dff82e5a7ce05ad3838ddefd"
Jul 09 18:09:28.462373 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.462175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b478b456-92c5q" event={"ID":"5b744141-9725-4d36-a8d7-551dc52d6b07","Type":"ContainerStarted","Data":"b61de603f146176a1aecb4ffe6677c28eacba9431a5861d1fabf7ee83e3f6681"}
Jul 09 18:09:28.464231 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.464207    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fk6jw" event={"ID":"e30217fd-44c2-41e6-b154-6e9d2e35ce6f","Type":"ContainerStarted","Data":"bcecfb5754d973303f46a7d583d6c03814519618bc0908e4e607e28a22b8c2ef"}
Jul 09 18:09:28.464405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.464383    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:28.478517 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.478483    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f54f75b7c-874l5"]
Jul 09 18:09:28.480558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.480535    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-fk6jw"
Jul 09 18:09:28.481908 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.481887    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f54f75b7c-874l5"]
Jul 09 18:09:28.498390 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.498346    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b478b456-92c5q" podStartSLOduration=2.042315095 podStartE2EDuration="11.498326465s" totalImagesPullingTime="9.45601137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:09:28.495860333 +0000 UTC m=+330.842768275" watchObservedRunningTime="2026-07-09 18:09:28.498326465 +0000 UTC m=+330.845234384"
Jul 09 18:09:28.512010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:28.511961    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-fk6jw" podStartSLOduration=1.438834414 podStartE2EDuration="20.511941501s" totalImagesPullingTime="19.073107087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:09:28.510690531 +0000 UTC m=+330.857598448" watchObservedRunningTime="2026-07-09 18:09:28.511941501 +0000 UTC m=+330.858849420"
Jul 09 18:09:30.195061 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:30.195031    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" path="/var/lib/kubelet/pods/0ac7d7af-a5b1-4814-8edb-8f3472e3beb8/volumes"
Jul 09 18:09:32.382044 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:32.382002    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:32.482157 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:32.482122    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:33.492905 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:33.492865    2567 generic.go:356] "Generic (PLEG): container finished" podID="a189a918-de0e-49b0-a269-79ef6f699c06" containerID="579cc78a8ba3d918bf32d626f08985c116471f717e540547c8e21353071f0a2f" exitCode=0
Jul 09 18:09:33.493422 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:33.492926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp" event={"ID":"a189a918-de0e-49b0-a269-79ef6f699c06","Type":"ContainerDied","Data":"579cc78a8ba3d918bf32d626f08985c116471f717e540547c8e21353071f0a2f"}
Jul 09 18:09:33.493422 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:33.493320    2567 scope.go:117] "RemoveContainer" containerID="579cc78a8ba3d918bf32d626f08985c116471f717e540547c8e21353071f0a2f"
Jul 09 18:09:33.494666 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:33.494639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:33.595664 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:33.595630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:34.501405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:34.501362    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lplzp" event={"ID":"a189a918-de0e-49b0-a269-79ef6f699c06","Type":"ContainerStarted","Data":"fa43754cd56354373f4d56b1b19d8dad87a2cb2e3606bce73840124cb0ce9751"}
Jul 09 18:09:35.076427 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.076386    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/init-config-reloader/0.log"
Jul 09 18:09:35.273951 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.273911    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/alertmanager/0.log"
Jul 09 18:09:35.472417 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.472355    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/config-reloader/0.log"
Jul 09 18:09:35.540581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.540545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:35.647655 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.647624    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:09:35.652041 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.652016    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-58c7786f4b-k57cq"
Jul 09 18:09:35.673690 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.673663    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/kube-rbac-proxy-web/0.log"
Jul 09 18:09:35.873529 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:35.873452    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/kube-rbac-proxy/0.log"
Jul 09 18:09:36.073366 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.073335    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/kube-rbac-proxy-metric/0.log"
Jul 09 18:09:36.272580 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.272542    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c91103d5-115a-4385-97a6-ceeae55d2483/prom-label-proxy/0.log"
Jul 09 18:09:36.331593 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.331557    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b478b456-92c5q"]
Jul 09 18:09:36.465430 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.465399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:36.479320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.479292    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-2qkks_e6c97669-d87d-4e2f-b35c-c7a39853020c/cluster-monitoring-operator/0.log"
Jul 09 18:09:36.555354 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.555252    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:36.655183 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:36.655141    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:37.273216 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:37.273175    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58c7786f4b-k57cq_203d4f89-7850-46c7-952d-c6c8c0730ec0/metrics-server/0.log"
Jul 09 18:09:37.673773 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:37.673674    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-47fhv_114cd342-a369-4587-bf22-786756baa888/init-textfile/0.log"
Jul 09 18:09:37.874281 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:37.874248    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-47fhv_114cd342-a369-4587-bf22-786756baa888/node-exporter/0.log"
Jul 09 18:09:38.072812 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:38.072760    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-47fhv_114cd342-a369-4587-bf22-786756baa888/kube-rbac-proxy/0.log"
Jul 09 18:09:38.264096 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:38.264000    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:09:38.521631 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:38.521558    2567 generic.go:356] "Generic (PLEG): container finished" podID="9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc" containerID="498e8b8cee79d46be016a90a76acb593f0de015c78c1a164e354d01d03cf1e40" exitCode=0
Jul 09 18:09:38.521631 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:38.521600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l" event={"ID":"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc","Type":"ContainerDied","Data":"498e8b8cee79d46be016a90a76acb593f0de015c78c1a164e354d01d03cf1e40"}
Jul 09 18:09:38.521943 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:38.521929    2567 scope.go:117] "RemoveContainer" containerID="498e8b8cee79d46be016a90a76acb593f0de015c78c1a164e354d01d03cf1e40"
Jul 09 18:09:39.473457 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:39.473422    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s68lq_21121af4-c2c8-4f75-880a-b33e78df782b/kube-rbac-proxy-main/0.log"
Jul 09 18:09:39.528117 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:39.528082    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q7z4l" event={"ID":"9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc","Type":"ContainerStarted","Data":"68049c0f1bb4c12fe2eb57158d882bba67512ebf7a66d845ef20c959e1dc4445"}
Jul 09 18:09:39.673861 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:39.673830    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s68lq_21121af4-c2c8-4f75-880a-b33e78df782b/kube-rbac-proxy-self/0.log"
Jul 09 18:09:39.872725 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:39.872633    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s68lq_21121af4-c2c8-4f75-880a-b33e78df782b/openshift-state-metrics/0.log"
Jul 09 18:09:41.475612 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:41.475573    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-c59fq_5c0c272e-4bcc-4608-b033-7413930c4dda/prometheus-operator/0.log"
Jul 09 18:09:41.672669 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:41.672630    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-c59fq_5c0c272e-4bcc-4608-b033-7413930c4dda/kube-rbac-proxy/0.log"
Jul 09 18:09:41.800320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:41.800235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:41.872560 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:41.872520    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-xxt4p_3d0b072b-ad0d-450a-a008-1f4094360d95/prometheus-operator-admission-webhook/0.log"
Jul 09 18:09:41.900978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:41.900948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:41.953616 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:41.953586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:42.073218 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:42.073136    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66dbd5c75-zxkf9_ffb23c53-46ab-47e3-b760-f5bd4e30019f/telemeter-client/0.log"
Jul 09 18:09:42.273477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:42.273444    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66dbd5c75-zxkf9_ffb23c53-46ab-47e3-b760-f5bd4e30019f/reload/0.log"
Jul 09 18:09:42.472493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:42.472451    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66dbd5c75-zxkf9_ffb23c53-46ab-47e3-b760-f5bd4e30019f/kube-rbac-proxy/0.log"
Jul 09 18:09:42.673688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:42.673654    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/thanos-query/0.log"
Jul 09 18:09:42.873320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:42.873235    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy-web/0.log"
Jul 09 18:09:43.073198 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:43.073173    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy/0.log"
Jul 09 18:09:43.273557 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:43.273529    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/prom-label-proxy/0.log"
Jul 09 18:09:43.473073 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:43.473047    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy-rules/0.log"
Jul 09 18:09:43.674767 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:43.674681    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy-metrics/0.log"
Jul 09 18:09:43.873278 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:43.873242    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-hgdbm_8bbaae80-06b4-43bf-92f1-def9f70426f4/networking-console-plugin/0.log"
Jul 09 18:09:44.473505 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:44.473477    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77b478b456-92c5q_5b744141-9725-4d36-a8d7-551dc52d6b07/console/0.log"
Jul 09 18:09:44.875085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:44.875000    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fk6jw_e30217fd-44c2-41e6-b154-6e9d2e35ce6f/download-server/0.log"
Jul 09 18:09:45.477485 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:45.477450    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hnmt_0236ff24-a897-4d0f-b0b0-23d3a7904f2b/serve-healthcheck-canary/0.log"
Jul 09 18:09:47.373842 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:47.373804    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:47.474535 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:09:47.474500    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:01.366839 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.366763    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b478b456-92c5q" podUID="5b744141-9725-4d36-a8d7-551dc52d6b07" containerName="console" containerID="cri-o://b61de603f146176a1aecb4ffe6677c28eacba9431a5861d1fabf7ee83e3f6681" gracePeriod=15
Jul 09 18:10:01.633175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.633144    2567 generic.go:356] "Generic (PLEG): container finished" podID="5b744141-9725-4d36-a8d7-551dc52d6b07" containerID="b61de603f146176a1aecb4ffe6677c28eacba9431a5861d1fabf7ee83e3f6681" exitCode=0
Jul 09 18:10:01.633319 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.633199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b478b456-92c5q" event={"ID":"5b744141-9725-4d36-a8d7-551dc52d6b07","Type":"ContainerDied","Data":"b61de603f146176a1aecb4ffe6677c28eacba9431a5861d1fabf7ee83e3f6681"}
Jul 09 18:10:01.633319 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.633243    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b478b456-92c5q" event={"ID":"5b744141-9725-4d36-a8d7-551dc52d6b07","Type":"ContainerDied","Data":"3dd2947c725d5a1f02df0ab5f78106337fa73aad12a6656b3145f981d74c27ce"}
Jul 09 18:10:01.633319 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.633263    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3dd2947c725d5a1f02df0ab5f78106337fa73aad12a6656b3145f981d74c27ce"
Jul 09 18:10:01.653149 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.653116    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:10:01.665748 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.665718    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-service-ca\") pod \"5b744141-9725-4d36-a8d7-551dc52d6b07\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") "
Jul 09 18:10:01.665910 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.665755    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-serving-cert\") pod \"5b744141-9725-4d36-a8d7-551dc52d6b07\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") "
Jul 09 18:10:01.665910 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.665819    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7xxh\" (UniqueName: \"kubernetes.io/projected/5b744141-9725-4d36-a8d7-551dc52d6b07-kube-api-access-j7xxh\") pod \"5b744141-9725-4d36-a8d7-551dc52d6b07\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") "
Jul 09 18:10:01.665910 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.665863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-oauth-config\") pod \"5b744141-9725-4d36-a8d7-551dc52d6b07\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") "
Jul 09 18:10:01.665910 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.665904    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-console-config\") pod \"5b744141-9725-4d36-a8d7-551dc52d6b07\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") "
Jul 09 18:10:01.666127 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.665923    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-oauth-serving-cert\") pod \"5b744141-9725-4d36-a8d7-551dc52d6b07\" (UID: \"5b744141-9725-4d36-a8d7-551dc52d6b07\") "
Jul 09 18:10:01.666326 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.666201    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-service-ca" (OuterVolumeSpecName: "service-ca") pod "5b744141-9725-4d36-a8d7-551dc52d6b07" (UID: "5b744141-9725-4d36-a8d7-551dc52d6b07"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:10:01.666326 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.666299    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-console-config" (OuterVolumeSpecName: "console-config") pod "5b744141-9725-4d36-a8d7-551dc52d6b07" (UID: "5b744141-9725-4d36-a8d7-551dc52d6b07"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:10:01.666489 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.666342    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5b744141-9725-4d36-a8d7-551dc52d6b07" (UID: "5b744141-9725-4d36-a8d7-551dc52d6b07"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:10:01.668173 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.668137    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5b744141-9725-4d36-a8d7-551dc52d6b07" (UID: "5b744141-9725-4d36-a8d7-551dc52d6b07"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:01.668257 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.668178    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b744141-9725-4d36-a8d7-551dc52d6b07-kube-api-access-j7xxh" (OuterVolumeSpecName: "kube-api-access-j7xxh") pod "5b744141-9725-4d36-a8d7-551dc52d6b07" (UID: "5b744141-9725-4d36-a8d7-551dc52d6b07"). InnerVolumeSpecName "kube-api-access-j7xxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:10:01.668257 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.668228    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5b744141-9725-4d36-a8d7-551dc52d6b07" (UID: "5b744141-9725-4d36-a8d7-551dc52d6b07"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:01.766853 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.766816    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-console-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:01.766853 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.766847    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-oauth-serving-cert\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:01.766853 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.766856    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b744141-9725-4d36-a8d7-551dc52d6b07-service-ca\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:01.766853 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.766866    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-serving-cert\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:01.767124 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.766875    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7xxh\" (UniqueName: \"kubernetes.io/projected/5b744141-9725-4d36-a8d7-551dc52d6b07-kube-api-access-j7xxh\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:01.767124 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:01.766885    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b744141-9725-4d36-a8d7-551dc52d6b07-console-oauth-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:02.637550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:02.637466    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b478b456-92c5q"
Jul 09 18:10:02.653986 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:02.653953    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b478b456-92c5q"]
Jul 09 18:10:02.657356 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:02.657329    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b478b456-92c5q"]
Jul 09 18:10:04.193713 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:04.193679    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b744141-9725-4d36-a8d7-551dc52d6b07" path="/var/lib/kubelet/pods/5b744141-9725-4d36-a8d7-551dc52d6b07/volumes"
Jul 09 18:10:06.974242 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:06.974205    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:07.074586 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:07.074544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:08.378520 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378487    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5669459db9-2qvpl"]
Jul 09 18:10:08.378906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378866    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" containerName="registry"
Jul 09 18:10:08.378906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378879    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" containerName="registry"
Jul 09 18:10:08.378906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378894    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b744141-9725-4d36-a8d7-551dc52d6b07" containerName="console"
Jul 09 18:10:08.378906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378900    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b744141-9725-4d36-a8d7-551dc52d6b07" containerName="console"
Jul 09 18:10:08.379037 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378961    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac7d7af-a5b1-4814-8edb-8f3472e3beb8" containerName="registry"
Jul 09 18:10:08.379037 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.378969    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b744141-9725-4d36-a8d7-551dc52d6b07" containerName="console"
Jul 09 18:10:08.408162 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.408140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5669459db9-2qvpl"]
Jul 09 18:10:08.408294 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.408235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.411966 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.411099    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 18:10:08.411966 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.411498    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 18:10:08.411966 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.411513    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 18:10:08.411966 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.411798    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 18:10:08.411966 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.411830    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7bkfr\""
Jul 09 18:10:08.414578 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.414558    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 18:10:08.419130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.419107    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 18:10:08.470768 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.470732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:08.520089 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-service-ca\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.520089 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-config\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.520265 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-trusted-ca-bundle\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.520265 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520239    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-oauth-serving-cert\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.520341 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-serving-cert\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.520341 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-oauth-config\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.520341 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.520336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6pg\" (UniqueName: \"kubernetes.io/projected/9710a30f-56f3-4fd4-a91a-ed7484861b54-kube-api-access-ht6pg\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.571651 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.571617    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:08.621324 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-service-ca\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.621455 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-config\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.621455 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-trusted-ca-bundle\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.621600 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-oauth-serving-cert\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.621643 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-serving-cert\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.621693 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-oauth-config\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.621900 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.621810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6pg\" (UniqueName: \"kubernetes.io/projected/9710a30f-56f3-4fd4-a91a-ed7484861b54-kube-api-access-ht6pg\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.622147 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.622118    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-service-ca\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.622260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.622175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-config\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.622260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.622218    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-trusted-ca-bundle\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.622346 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.622261    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-oauth-serving-cert\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.624570 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.624546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-serving-cert\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.624683 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.624663    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-oauth-config\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.630344 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.630293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6pg\" (UniqueName: \"kubernetes.io/projected/9710a30f-56f3-4fd4-a91a-ed7484861b54-kube-api-access-ht6pg\") pod \"console-5669459db9-2qvpl\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") " pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.721156 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.721124    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:08.879373 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:08.879350    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5669459db9-2qvpl"]
Jul 09 18:10:08.882246 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:10:08.882182    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9710a30f_56f3_4fd4_a91a_ed7484861b54.slice/crio-19a0fba3fb5545740c869347e6a3d7975dcbf481d0aba3a78a2fc51754b6d581 WatchSource:0}: Error finding container 19a0fba3fb5545740c869347e6a3d7975dcbf481d0aba3a78a2fc51754b6d581: Status 404 returned error can't find the container with id 19a0fba3fb5545740c869347e6a3d7975dcbf481d0aba3a78a2fc51754b6d581
Jul 09 18:10:09.668697 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:09.668661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5669459db9-2qvpl" event={"ID":"9710a30f-56f3-4fd4-a91a-ed7484861b54","Type":"ContainerStarted","Data":"13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28"}
Jul 09 18:10:09.669102 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:09.668718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5669459db9-2qvpl" event={"ID":"9710a30f-56f3-4fd4-a91a-ed7484861b54","Type":"ContainerStarted","Data":"19a0fba3fb5545740c869347e6a3d7975dcbf481d0aba3a78a2fc51754b6d581"}
Jul 09 18:10:09.687638 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:09.687594    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5669459db9-2qvpl" podStartSLOduration=1.687580222 podStartE2EDuration="1.687580222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:10:09.685528097 +0000 UTC m=+372.032436039" watchObservedRunningTime="2026-07-09 18:10:09.687580222 +0000 UTC m=+372.034488138"
Jul 09 18:10:11.246484 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246452    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:10:11.247017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246735    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="alertmanager" containerID="cri-o://fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0" gracePeriod=120
Jul 09 18:10:11.247017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246834    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-web" containerID="cri-o://1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd" gracePeriod=120
Jul 09 18:10:11.247017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246850    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="config-reloader" containerID="cri-o://b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1" gracePeriod=120
Jul 09 18:10:11.247017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246856    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy" containerID="cri-o://7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761" gracePeriod=120
Jul 09 18:10:11.247017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246834    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-metric" containerID="cri-o://4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c" gracePeriod=120
Jul 09 18:10:11.247017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.246863    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="prom-label-proxy" containerID="cri-o://b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b" gracePeriod=120
Jul 09 18:10:11.680188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680103    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b" exitCode=0
Jul 09 18:10:11.680188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680128    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c" exitCode=0
Jul 09 18:10:11.680188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680135    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761" exitCode=0
Jul 09 18:10:11.680188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680140    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1" exitCode=0
Jul 09 18:10:11.680188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680145    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0" exitCode=0
Jul 09 18:10:11.680188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680168    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b"}
Jul 09 18:10:11.680477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c"}
Jul 09 18:10:11.680477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680261    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761"}
Jul 09 18:10:11.680477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680286    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1"}
Jul 09 18:10:11.680477 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:11.680309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0"}
Jul 09 18:10:12.052271 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.052243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:10:12.054671 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.054648    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcfc3b54-df61-4210-b823-da5675e01562-metrics-certs\") pod \"network-metrics-daemon-nm5xr\" (UID: \"bcfc3b54-df61-4210-b823-da5675e01562\") " pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:10:12.293072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.293033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7k8ns\""
Jul 09 18:10:12.300558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.300530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm5xr"
Jul 09 18:10:12.446496 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.446470    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nm5xr"]
Jul 09 18:10:12.448757 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:10:12.448727    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfc3b54_df61_4210_b823_da5675e01562.slice/crio-f2203143a6b32c7fdea3a16a9e46d4dca0f97a691aabb07f2aae5a23801c6de8 WatchSource:0}: Error finding container f2203143a6b32c7fdea3a16a9e46d4dca0f97a691aabb07f2aae5a23801c6de8: Status 404 returned error can't find the container with id f2203143a6b32c7fdea3a16a9e46d4dca0f97a691aabb07f2aae5a23801c6de8
Jul 09 18:10:12.522162 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.522133    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.556801 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.556752    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k687q\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-kube-api-access-k687q\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.556999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.556864    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.556999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.556911    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-tls-assets\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557137 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.556994    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557137 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557073    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-web-config\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557137 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557108    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-main-db\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557291 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557144    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557291 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557180    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-config-out\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557291 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557219    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-metrics-client-ca\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557291 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557271    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-cluster-tls-config\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557300    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:10:12.557493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557319    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-config-volume\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557343    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.557493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557386    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-web\") pod \"c91103d5-115a-4385-97a6-ceeae55d2483\" (UID: \"c91103d5-115a-4385-97a6-ceeae55d2483\") "
Jul 09 18:10:12.559077 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557727    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.559077 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.557945    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:10:12.559077 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.558705    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:10:12.559632 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.559605    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.560125 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.560098    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-kube-api-access-k687q" (OuterVolumeSpecName: "kube-api-access-k687q") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "kube-api-access-k687q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:10:12.561106 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.561078    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.562191 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.562163    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-config-volume" (OuterVolumeSpecName: "config-volume") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.562317 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.562294    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-config-out" (OuterVolumeSpecName: "config-out") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:10:12.562455 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.562431    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.562916 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.562890    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:10:12.563083 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.562960    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.566042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.565993    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.575912 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.575888    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-web-config" (OuterVolumeSpecName: "web-config") pod "c91103d5-115a-4385-97a6-ceeae55d2483" (UID: "c91103d5-115a-4385-97a6-ceeae55d2483"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:10:12.658391 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658354    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-tls-assets\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658391 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658386    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-main-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658417    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-web-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658426    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-alertmanager-main-db\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658436    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658445    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c91103d5-115a-4385-97a6-ceeae55d2483-config-out\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658454    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91103d5-115a-4385-97a6-ceeae55d2483-metrics-client-ca\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658463    2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-cluster-tls-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658471    2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-config-volume\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658480    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658490    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c91103d5-115a-4385-97a6-ceeae55d2483-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.658619 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.658500    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k687q\" (UniqueName: \"kubernetes.io/projected/c91103d5-115a-4385-97a6-ceeae55d2483-kube-api-access-k687q\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:10:12.686345 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.686310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nm5xr" event={"ID":"bcfc3b54-df61-4210-b823-da5675e01562","Type":"ContainerStarted","Data":"f2203143a6b32c7fdea3a16a9e46d4dca0f97a691aabb07f2aae5a23801c6de8"}
Jul 09 18:10:12.689211 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.689179    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91103d5-115a-4385-97a6-ceeae55d2483" containerID="1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd" exitCode=0
Jul 09 18:10:12.689316 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.689214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd"}
Jul 09 18:10:12.689316 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.689261    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.689316 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.689274    2567 scope.go:117] "RemoveContainer" containerID="b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b"
Jul 09 18:10:12.689433 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.689264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c91103d5-115a-4385-97a6-ceeae55d2483","Type":"ContainerDied","Data":"228cccbe08a7299ce61e49b996f0351862a3135490de355b04260422dd86ed0a"}
Jul 09 18:10:12.697165 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.697143    2567 scope.go:117] "RemoveContainer" containerID="4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c"
Jul 09 18:10:12.704611 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.704596    2567 scope.go:117] "RemoveContainer" containerID="7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761"
Jul 09 18:10:12.712420 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.712403    2567 scope.go:117] "RemoveContainer" containerID="1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd"
Jul 09 18:10:12.713031 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.713010    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:10:12.717694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.717670    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:10:12.719907 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.719889    2567 scope.go:117] "RemoveContainer" containerID="b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1"
Jul 09 18:10:12.726499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.726482    2567 scope.go:117] "RemoveContainer" containerID="fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0"
Jul 09 18:10:12.733377 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.733358    2567 scope.go:117] "RemoveContainer" containerID="5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f"
Jul 09 18:10:12.740496 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.740477    2567 scope.go:117] "RemoveContainer" containerID="b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b"
Jul 09 18:10:12.740769 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.740740    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b\": container with ID starting with b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b not found: ID does not exist" containerID="b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b"
Jul 09 18:10:12.740873 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.740796    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b"} err="failed to get container status \"b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b\": rpc error: code = NotFound desc = could not find container \"b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b\": container with ID starting with b597a79f8285cd3c9c47d43357740a355cd7280c2d254c267d014d18cb0dd97b not found: ID does not exist"
Jul 09 18:10:12.740873 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.740841    2567 scope.go:117] "RemoveContainer" containerID="4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c"
Jul 09 18:10:12.741166 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.741147    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c\": container with ID starting with 4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c not found: ID does not exist" containerID="4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c"
Jul 09 18:10:12.741231 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.741173    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c"} err="failed to get container status \"4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c\": rpc error: code = NotFound desc = could not find container \"4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c\": container with ID starting with 4de7c8cd4cb396afc979b69f9a71791e320c69df995643d57bb1825d01cdcf1c not found: ID does not exist"
Jul 09 18:10:12.741231 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.741190    2567 scope.go:117] "RemoveContainer" containerID="7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761"
Jul 09 18:10:12.741503 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.741480    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761\": container with ID starting with 7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761 not found: ID does not exist" containerID="7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761"
Jul 09 18:10:12.741560 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.741512    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761"} err="failed to get container status \"7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761\": rpc error: code = NotFound desc = could not find container \"7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761\": container with ID starting with 7cf468e2bb5176db5c66dc5ef1cecdea36ae42cb00fda321d99a17c1cb016761 not found: ID does not exist"
Jul 09 18:10:12.741560 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.741545    2567 scope.go:117] "RemoveContainer" containerID="1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd"
Jul 09 18:10:12.741950 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.741925    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd\": container with ID starting with 1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd not found: ID does not exist" containerID="1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd"
Jul 09 18:10:12.742022 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.741997    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd"} err="failed to get container status \"1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd\": rpc error: code = NotFound desc = could not find container \"1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd\": container with ID starting with 1eba48c7b2a796d5624c690c611c6bb1d55d869748fdb5e8a29eaee21e93febd not found: ID does not exist"
Jul 09 18:10:12.742022 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.742017    2567 scope.go:117] "RemoveContainer" containerID="b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1"
Jul 09 18:10:12.742318 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.742299    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1\": container with ID starting with b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1 not found: ID does not exist" containerID="b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1"
Jul 09 18:10:12.742367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.742324    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1"} err="failed to get container status \"b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1\": rpc error: code = NotFound desc = could not find container \"b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1\": container with ID starting with b476343bc264e1e00fc265cb8fb692e65ca5c31c89cdafa314fb2bfd5a8386d1 not found: ID does not exist"
Jul 09 18:10:12.742367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.742343    2567 scope.go:117] "RemoveContainer" containerID="fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0"
Jul 09 18:10:12.742595 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.742576    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0\": container with ID starting with fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0 not found: ID does not exist" containerID="fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0"
Jul 09 18:10:12.742646 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.742610    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0"} err="failed to get container status \"fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0\": rpc error: code = NotFound desc = could not find container \"fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0\": container with ID starting with fdfed45b29cb5a7ed67175969c563daa6559f047e48f3ee0d90c37160306dce0 not found: ID does not exist"
Jul 09 18:10:12.742646 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.742626    2567 scope.go:117] "RemoveContainer" containerID="5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f"
Jul 09 18:10:12.742835 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:10:12.742819    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f\": container with ID starting with 5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f not found: ID does not exist" containerID="5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f"
Jul 09 18:10:12.742882 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.742838    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f"} err="failed to get container status \"5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f\": rpc error: code = NotFound desc = could not find container \"5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f\": container with ID starting with 5aecb1dc20b74c45a36d6c4775cdad4e9934b4a9f005f6c6f8919b1af8364a5f not found: ID does not exist"
Jul 09 18:10:12.743234 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743218    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:10:12.743538 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743528    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-web"
Jul 09 18:10:12.743575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743540    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-web"
Jul 09 18:10:12.743575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743551    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="config-reloader"
Jul 09 18:10:12.743575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743557    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="config-reloader"
Jul 09 18:10:12.743575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743564    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="alertmanager"
Jul 09 18:10:12.743575 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743569    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="alertmanager"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743582    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-metric"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743587    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-metric"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743600    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="init-config-reloader"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743606    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="init-config-reloader"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743611    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="prom-label-proxy"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743616    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="prom-label-proxy"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743624    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743628    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743679    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="config-reloader"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743688    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-metric"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743693    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743701    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="kube-rbac-proxy-web"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743708    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="alertmanager"
Jul 09 18:10:12.743726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.743713    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" containerName="prom-label-proxy"
Jul 09 18:10:12.770953 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.770934    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:10:12.771073 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.771061    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.773946 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.773925    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 18:10:12.773946 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.773955    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 18:10:12.774399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.773969    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dwlw2\""
Jul 09 18:10:12.774399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.774234    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 18:10:12.774399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.774393    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 18:10:12.774572 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.774483    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 18:10:12.774759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.774743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 18:10:12.774825 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.774764    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 18:10:12.774825 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.774743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 18:10:12.780889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.780871    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 18:10:12.860960 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.860874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.860960 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.860921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.860963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.860993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861017    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ca9a665-cfcc-4541-8041-fd5822781959-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ca9a665-cfcc-4541-8041-fd5822781959-config-out\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861073    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca9a665-cfcc-4541-8041-fd5822781959-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861194 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861114    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-web-config\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ca9a665-cfcc-4541-8041-fd5822781959-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ca9a665-cfcc-4541-8041-fd5822781959-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861306    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd6x6\" (UniqueName: \"kubernetes.io/projected/1ca9a665-cfcc-4541-8041-fd5822781959-kube-api-access-bd6x6\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.861432 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.861350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.961874 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.961831    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962058 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.961887    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962058 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962058 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ca9a665-cfcc-4541-8041-fd5822781959-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962219 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ca9a665-cfcc-4541-8041-fd5822781959-config-out\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962219 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca9a665-cfcc-4541-8041-fd5822781959-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962219 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962219 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962199    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-web-config\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ca9a665-cfcc-4541-8041-fd5822781959-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ca9a665-cfcc-4541-8041-fd5822781959-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962297    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd6x6\" (UniqueName: \"kubernetes.io/projected/1ca9a665-cfcc-4541-8041-fd5822781959-kube-api-access-bd6x6\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962342    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.962640 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.962508    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ca9a665-cfcc-4541-8041-fd5822781959-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.964453 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.963455    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca9a665-cfcc-4541-8041-fd5822781959-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.964453 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.964101    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ca9a665-cfcc-4541-8041-fd5822781959-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.965121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.964769    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ca9a665-cfcc-4541-8041-fd5822781959-config-out\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.965121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.964894    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.965121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.964963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.965121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.965061    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.965425 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.965132    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.965612 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.965567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.966876 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.966853    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.967303 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.967282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-web-config\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.967456 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.967439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ca9a665-cfcc-4541-8041-fd5822781959-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.967616 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.967595    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ca9a665-cfcc-4541-8041-fd5822781959-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:12.972458 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:12.972439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd6x6\" (UniqueName: \"kubernetes.io/projected/1ca9a665-cfcc-4541-8041-fd5822781959-kube-api-access-bd6x6\") pod \"alertmanager-main-0\" (UID: \"1ca9a665-cfcc-4541-8041-fd5822781959\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:13.081837 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:13.081798    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:10:13.248002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:13.247976    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:10:13.459650 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:10:13.459611    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca9a665_cfcc_4541_8041_fd5822781959.slice/crio-13b51563071edabed7aabdfee5487114c21b2eda49e8bf604d8679288e62375f WatchSource:0}: Error finding container 13b51563071edabed7aabdfee5487114c21b2eda49e8bf604d8679288e62375f: Status 404 returned error can't find the container with id 13b51563071edabed7aabdfee5487114c21b2eda49e8bf604d8679288e62375f
Jul 09 18:10:13.695490 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:13.695457    2567 generic.go:356] "Generic (PLEG): container finished" podID="1ca9a665-cfcc-4541-8041-fd5822781959" containerID="8f0e091dd1ae6ff5d351d3a66a6dd039ce3242e6db5675357ffd1f69269b0939" exitCode=0
Jul 09 18:10:13.695654 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:13.695547    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerDied","Data":"8f0e091dd1ae6ff5d351d3a66a6dd039ce3242e6db5675357ffd1f69269b0939"}
Jul 09 18:10:13.695654 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:13.695629    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"13b51563071edabed7aabdfee5487114c21b2eda49e8bf604d8679288e62375f"}
Jul 09 18:10:14.199759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.199732    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c91103d5-115a-4385-97a6-ceeae55d2483" path="/var/lib/kubelet/pods/c91103d5-115a-4385-97a6-ceeae55d2483/volumes"
Jul 09 18:10:14.703676 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.703641    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nm5xr" event={"ID":"bcfc3b54-df61-4210-b823-da5675e01562","Type":"ContainerStarted","Data":"4f680e738cb264dee8c7b386a423fcbfe7c1dda9f2e837d9d70656c6532ee0fd"}
Jul 09 18:10:14.704137 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.703695    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nm5xr" event={"ID":"bcfc3b54-df61-4210-b823-da5675e01562","Type":"ContainerStarted","Data":"06bddcae5e517b7803575d5e0e43cc6ff0ba98899044524d7f1f66c07a74ab5c"}
Jul 09 18:10:14.707065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.707039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"f1bf08dd3b9b262867dde9af6c12e01258362c9ad70ebc72c4fc0fb337b2ae24"}
Jul 09 18:10:14.707214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.707104    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"81378eebae0b6124a5c899a66310f410982c47a16d56d81435ee5be7778ab153"}
Jul 09 18:10:14.707214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.707150    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"65985c7d7072fc70621cdb921016334a2db0a668f09c1b5a4323bb4209387093"}
Jul 09 18:10:14.707214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.707189    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"5d7f15671f90ba00ed1265c35a860d4f69ee105b209bcc6b76ae565f59e66f52"}
Jul 09 18:10:14.707351 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.707227    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"bab5be4709fce300a8543c60ce934a11d91ea2310997abf386b0672416c88968"}
Jul 09 18:10:14.707351 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.707265    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ca9a665-cfcc-4541-8041-fd5822781959","Type":"ContainerStarted","Data":"f4fd2d0d1fef561ac9b1bd4770cb454c3c842fff823bc6387d8b00b4940b79e0"}
Jul 09 18:10:14.720887 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.720844    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nm5xr" podStartSLOduration=375.336185936 podStartE2EDuration="6m16.720831695s" totalImagesPullingTime="1.384645759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:10:14.719296361 +0000 UTC m=+377.066204302" watchObservedRunningTime="2026-07-09 18:10:14.720831695 +0000 UTC m=+377.067739611"
Jul 09 18:10:14.750364 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.750312    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.731032898 podStartE2EDuration="2.750299701s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.266803ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:10:14.748630586 +0000 UTC m=+377.095538503" watchObservedRunningTime="2026-07-09 18:10:14.750299701 +0000 UTC m=+377.097207618"
Jul 09 18:10:14.872303 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.872268    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:14.926773 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:14.926738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:18.722147 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:18.722104    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:18.722599 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:18.722168    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:18.727032 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:18.727009    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:18.823136 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:18.823102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:18.882962 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:18.882922    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:19.734534 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:19.734505    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:10:19.898583 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:19.898550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:19.999606 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:19.999513    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:20.074297 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:20.074266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:21.544533 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:21.544496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:21.609450 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:21.609409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:32.398704 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:32.398623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:32.499115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:10:32.499079    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:04.456725 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:04.456685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:04.512311 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:04.512278    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:15.991182 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:15.991143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:16.049803 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:16.049756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:16.378410 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:16.378316    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:16.480253 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:16.480221    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:16.586808 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:16.586765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:17.445256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.445223    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"]
Jul 09 18:11:17.449350 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.449331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.451719 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.451693    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 18:11:17.451858 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.451757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4nv\""
Jul 09 18:11:17.451858 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.451823    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 18:11:17.456676 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.456402    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"]
Jul 09 18:11:17.503588 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.503558    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.503742 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.503602    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2kx2\" (UniqueName: \"kubernetes.io/projected/5493f8eb-ea61-4081-b782-25762b839ab6-kube-api-access-x2kx2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.503742 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.503663    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.604222 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.604186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.604387 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.604263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.604387 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.604292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2kx2\" (UniqueName: \"kubernetes.io/projected/5493f8eb-ea61-4081-b782-25762b839ab6-kube-api-access-x2kx2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.604616 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.604594    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.604653 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.604626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.612471 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.612445    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2kx2\" (UniqueName: \"kubernetes.io/projected/5493f8eb-ea61-4081-b782-25762b839ab6-kube-api-access-x2kx2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.759352 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.759268    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:17.889151 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.889125    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"]
Jul 09 18:11:17.891980 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:11:17.891956    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5493f8eb_ea61_4081_b782_25762b839ab6.slice/crio-e9d42cb5da1610dbc9b4b0aebf7200abc07da20d7e01ac10c5e933c37e2bf459 WatchSource:0}: Error finding container e9d42cb5da1610dbc9b4b0aebf7200abc07da20d7e01ac10c5e933c37e2bf459: Status 404 returned error can't find the container with id e9d42cb5da1610dbc9b4b0aebf7200abc07da20d7e01ac10c5e933c37e2bf459
Jul 09 18:11:17.995352 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:17.995320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr" event={"ID":"5493f8eb-ea61-4081-b782-25762b839ab6","Type":"ContainerStarted","Data":"e9d42cb5da1610dbc9b4b0aebf7200abc07da20d7e01ac10c5e933c37e2bf459"}
Jul 09 18:11:20.077192 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:20.077103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:26.040684 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:26.040648    2567 generic.go:356] "Generic (PLEG): container finished" podID="5493f8eb-ea61-4081-b782-25762b839ab6" containerID="9a56294fa0042df1903652ce80c5918e29b4ae0fb813530997a4040525058fff" exitCode=0
Jul 09 18:11:26.041073 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:26.040729    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr" event={"ID":"5493f8eb-ea61-4081-b782-25762b839ab6","Type":"ContainerDied","Data":"9a56294fa0042df1903652ce80c5918e29b4ae0fb813530997a4040525058fff"}
Jul 09 18:11:29.059121 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:29.059088    2567 generic.go:356] "Generic (PLEG): container finished" podID="5493f8eb-ea61-4081-b782-25762b839ab6" containerID="0de02c15f85523fe88d90a51a4839ada4a44c95a80f2c0df92c11a1e5bfdfd84" exitCode=0
Jul 09 18:11:29.059552 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:29.059182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr" event={"ID":"5493f8eb-ea61-4081-b782-25762b839ab6","Type":"ContainerDied","Data":"0de02c15f85523fe88d90a51a4839ada4a44c95a80f2c0df92c11a1e5bfdfd84"}
Jul 09 18:11:38.107855 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:38.107820    2567 generic.go:356] "Generic (PLEG): container finished" podID="5493f8eb-ea61-4081-b782-25762b839ab6" containerID="e815e1c6ea9bc982cc0b069f22a06ee4aa43b0fbbcc0f2674029148c77f5e570" exitCode=0
Jul 09 18:11:38.108247 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:38.107897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr" event={"ID":"5493f8eb-ea61-4081-b782-25762b839ab6","Type":"ContainerDied","Data":"e815e1c6ea9bc982cc0b069f22a06ee4aa43b0fbbcc0f2674029148c77f5e570"}
Jul 09 18:11:39.239717 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.239694    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:39.301747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.301716    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-bundle\") pod \"5493f8eb-ea61-4081-b782-25762b839ab6\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") "
Jul 09 18:11:39.301967 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.301762    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-util\") pod \"5493f8eb-ea61-4081-b782-25762b839ab6\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") "
Jul 09 18:11:39.301967 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.301805    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2kx2\" (UniqueName: \"kubernetes.io/projected/5493f8eb-ea61-4081-b782-25762b839ab6-kube-api-access-x2kx2\") pod \"5493f8eb-ea61-4081-b782-25762b839ab6\" (UID: \"5493f8eb-ea61-4081-b782-25762b839ab6\") "
Jul 09 18:11:39.302323 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.302298    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-bundle" (OuterVolumeSpecName: "bundle") pod "5493f8eb-ea61-4081-b782-25762b839ab6" (UID: "5493f8eb-ea61-4081-b782-25762b839ab6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:11:39.304001 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.303973    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5493f8eb-ea61-4081-b782-25762b839ab6-kube-api-access-x2kx2" (OuterVolumeSpecName: "kube-api-access-x2kx2") pod "5493f8eb-ea61-4081-b782-25762b839ab6" (UID: "5493f8eb-ea61-4081-b782-25762b839ab6"). InnerVolumeSpecName "kube-api-access-x2kx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:11:39.306203 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.306178    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-util" (OuterVolumeSpecName: "util") pod "5493f8eb-ea61-4081-b782-25762b839ab6" (UID: "5493f8eb-ea61-4081-b782-25762b839ab6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:11:39.402776 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.402687    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-bundle\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:11:39.402776 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.402722    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5493f8eb-ea61-4081-b782-25762b839ab6-util\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:11:39.402776 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:39.402737    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2kx2\" (UniqueName: \"kubernetes.io/projected/5493f8eb-ea61-4081-b782-25762b839ab6-kube-api-access-x2kx2\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:11:40.118136 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:40.118109    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr"
Jul 09 18:11:40.118335 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:40.118112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb5898thbr" event={"ID":"5493f8eb-ea61-4081-b782-25762b839ab6","Type":"ContainerDied","Data":"e9d42cb5da1610dbc9b4b0aebf7200abc07da20d7e01ac10c5e933c37e2bf459"}
Jul 09 18:11:40.118335 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:40.118266    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9d42cb5da1610dbc9b4b0aebf7200abc07da20d7e01ac10c5e933c37e2bf459"
Jul 09 18:11:43.670927 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:43.670896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:43.769427 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:43.769395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:45.301203 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301170    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"]
Jul 09 18:11:45.301567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301499    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="extract"
Jul 09 18:11:45.301567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301509    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="extract"
Jul 09 18:11:45.301567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301518    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="util"
Jul 09 18:11:45.301567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301524    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="util"
Jul 09 18:11:45.301567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301546    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="pull"
Jul 09 18:11:45.301567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301551    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="pull"
Jul 09 18:11:45.301750 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.301604    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5493f8eb-ea61-4081-b782-25762b839ab6" containerName="extract"
Jul 09 18:11:45.304651 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.304633    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.307259 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.307234    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 09 18:11:45.307369 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.307284    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-h6pwh\""
Jul 09 18:11:45.307369 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.307303    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 09 18:11:45.307489 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.307437    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 09 18:11:45.318805 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.318762    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"]
Jul 09 18:11:45.347156 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.347120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjnh4\" (UniqueName: \"kubernetes.io/projected/0e29e753-b4c3-4627-a48c-67e399d2e13f-kube-api-access-jjnh4\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-tfv95\" (UID: \"0e29e753-b4c3-4627-a48c-67e399d2e13f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.347316 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.347243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0e29e753-b4c3-4627-a48c-67e399d2e13f-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-tfv95\" (UID: \"0e29e753-b4c3-4627-a48c-67e399d2e13f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.448312 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.448276    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0e29e753-b4c3-4627-a48c-67e399d2e13f-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-tfv95\" (UID: \"0e29e753-b4c3-4627-a48c-67e399d2e13f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.448475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.448322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjnh4\" (UniqueName: \"kubernetes.io/projected/0e29e753-b4c3-4627-a48c-67e399d2e13f-kube-api-access-jjnh4\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-tfv95\" (UID: \"0e29e753-b4c3-4627-a48c-67e399d2e13f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.450638 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.450606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0e29e753-b4c3-4627-a48c-67e399d2e13f-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-tfv95\" (UID: \"0e29e753-b4c3-4627-a48c-67e399d2e13f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.458525 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.458502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjnh4\" (UniqueName: \"kubernetes.io/projected/0e29e753-b4c3-4627-a48c-67e399d2e13f-kube-api-access-jjnh4\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-tfv95\" (UID: \"0e29e753-b4c3-4627-a48c-67e399d2e13f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.614687 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.614600    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:45.745047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:45.744857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"]
Jul 09 18:11:45.748080 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:11:45.748043    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e29e753_b4c3_4627_a48c_67e399d2e13f.slice/crio-59ee2f051c9c57abfd275285ae2a72237d39d99210ea7635dd900884fac68a6d WatchSource:0}: Error finding container 59ee2f051c9c57abfd275285ae2a72237d39d99210ea7635dd900884fac68a6d: Status 404 returned error can't find the container with id 59ee2f051c9c57abfd275285ae2a72237d39d99210ea7635dd900884fac68a6d
Jul 09 18:11:46.147382 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:46.147347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95" event={"ID":"0e29e753-b4c3-4627-a48c-67e399d2e13f","Type":"ContainerStarted","Data":"59ee2f051c9c57abfd275285ae2a72237d39d99210ea7635dd900884fac68a6d"}
Jul 09 18:11:49.169706 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.169668    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95" event={"ID":"0e29e753-b4c3-4627-a48c-67e399d2e13f","Type":"ContainerStarted","Data":"6f93ad1f8249eea97e10c7698c5dc0b3d141c0782a5a3c42136d87c976ef0977"}
Jul 09 18:11:49.170195 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.169818    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:11:49.193255 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.193200    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95" podStartSLOduration=0.933005136 podStartE2EDuration="4.193182789s" totalImagesPullingTime="3.260177653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:49.191958379 +0000 UTC m=+471.538866296" watchObservedRunningTime="2026-07-09 18:11:49.193182789 +0000 UTC m=+471.540090705"
Jul 09 18:11:49.610550 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.610515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:49.954386 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.954346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:49.963442 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.963418    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-jfwtb"]
Jul 09 18:11:49.965681 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.965666    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:49.968875 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.968852    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-szds9\""
Jul 09 18:11:49.968986 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.968943    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 09 18:11:49.969306 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.969287    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 09 18:11:49.983304 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.983283    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-jfwtb"]
Jul 09 18:11:49.985306 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.985286    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-cabundle0\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:49.985404 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.985321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:49.985509 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:49.985486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m4w7\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-kube-api-access-9m4w7\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.086848 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.086814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m4w7\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-kube-api-access-9m4w7\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.087049 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.086857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-cabundle0\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.087049 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.086877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.087049 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.087011    2567 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 09 18:11:50.087049 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.087026    2567 secret.go:281] references non-existent secret key: ca.crt
Jul 09 18:11:50.087049 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.087033    2567 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 09 18:11:50.087049 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.087044    2567 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-jfwtb: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 09 18:11:50.087353 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.087094    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates podName:cd2c48c7-1e05-4d72-bf03-67387fab8ea1 nodeName:}" failed. No retries permitted until 2026-07-09 18:11:50.587078545 +0000 UTC m=+472.933986439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates") pod "keda-operator-68bff59c-jfwtb" (UID: "cd2c48c7-1e05-4d72-bf03-67387fab8ea1") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 09 18:11:50.087522 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.087503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-cabundle0\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.097590 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.097560    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m4w7\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-kube-api-access-9m4w7\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.160442 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.160395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:50.344767 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.344681    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:50.462535 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.462505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:50.591246 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.591210    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:50.591411 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.591375    2567 secret.go:281] references non-existent secret key: ca.crt
Jul 09 18:11:50.591411 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.591400    2567 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 09 18:11:50.591411 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.591411    2567 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-jfwtb: references non-existent secret key: ca.crt
Jul 09 18:11:50.591519 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.591486    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates podName:cd2c48c7-1e05-4d72-bf03-67387fab8ea1 nodeName:}" failed. No retries permitted until 2026-07-09 18:11:51.591466582 +0000 UTC m=+473.938374479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates") pod "keda-operator-68bff59c-jfwtb" (UID: "cd2c48c7-1e05-4d72-bf03-67387fab8ea1") : references non-existent secret key: ca.crt
Jul 09 18:11:50.741586 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.741547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:50.746417 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.746393    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-5268b"]
Jul 09 18:11:50.748730 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.748715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:50.750965 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.750948    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 09 18:11:50.761975 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.761951    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-5268b"]
Jul 09 18:11:50.792902 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.792874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-certificates\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:50.793042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.792919    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqwnx\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-kube-api-access-xqwnx\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:50.838329 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.838286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:50.893973 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.893939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqwnx\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-kube-api-access-xqwnx\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:50.894143 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.894082    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-certificates\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:50.894209 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.894195    2567 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 09 18:11:50.894249 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.894220    2567 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-5268b: secret "keda-admission-webhooks-certs" not found
Jul 09 18:11:50.894292 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:11:50.894286    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-certificates podName:6df91432-09b3-48a0-b7f2-a6fc29b1f9ff nodeName:}" failed. No retries permitted until 2026-07-09 18:11:51.394268143 +0000 UTC m=+473.741176043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-certificates") pod "keda-admission-5fbd5c4644-5268b" (UID: "6df91432-09b3-48a0-b7f2-a6fc29b1f9ff") : secret "keda-admission-webhooks-certs" not found
Jul 09 18:11:50.903654 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.903615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqwnx\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-kube-api-access-xqwnx\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:50.987241 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:50.987207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:51.206609 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.206568    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:51.399147 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.399114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-certificates\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:51.401537 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.401509    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6df91432-09b3-48a0-b7f2-a6fc29b1f9ff-certificates\") pod \"keda-admission-5fbd5c4644-5268b\" (UID: \"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff\") " pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:51.435957 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.435931    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:51.573011 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.572905    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:51.601172 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.601136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:51.603574 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.603550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd2c48c7-1e05-4d72-bf03-67387fab8ea1-certificates\") pod \"keda-operator-68bff59c-jfwtb\" (UID: \"cd2c48c7-1e05-4d72-bf03-67387fab8ea1\") " pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:51.659209 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.659163    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:11:51.672298 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.672266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:51.776129 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.776084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:51.798407 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.798381    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-5268b"]
Jul 09 18:11:51.800448 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:11:51.800425    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df91432_09b3_48a0_b7f2_a6fc29b1f9ff.slice/crio-e2e4428524336f383b32001381f1b2ada45ec8026f1b8830abfe7fc24c37f897 WatchSource:0}: Error finding container e2e4428524336f383b32001381f1b2ada45ec8026f1b8830abfe7fc24c37f897: Status 404 returned error can't find the container with id e2e4428524336f383b32001381f1b2ada45ec8026f1b8830abfe7fc24c37f897
Jul 09 18:11:51.920268 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:51.920233    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-jfwtb"]
Jul 09 18:11:51.923636 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:11:51.923610    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2c48c7_1e05_4d72_bf03_67387fab8ea1.slice/crio-e78446002b129b31989e8c76c6f727cb96c3acfe3d1586a57040c4a6fd894746 WatchSource:0}: Error finding container e78446002b129b31989e8c76c6f727cb96c3acfe3d1586a57040c4a6fd894746: Status 404 returned error can't find the container with id e78446002b129b31989e8c76c6f727cb96c3acfe3d1586a57040c4a6fd894746
Jul 09 18:11:52.186407 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:52.186315    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-jfwtb" event={"ID":"cd2c48c7-1e05-4d72-bf03-67387fab8ea1","Type":"ContainerStarted","Data":"e78446002b129b31989e8c76c6f727cb96c3acfe3d1586a57040c4a6fd894746"}
Jul 09 18:11:52.187327 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:52.187303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-5268b" event={"ID":"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff","Type":"ContainerStarted","Data":"e2e4428524336f383b32001381f1b2ada45ec8026f1b8830abfe7fc24c37f897"}
Jul 09 18:11:56.210809 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:56.210758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-jfwtb" event={"ID":"cd2c48c7-1e05-4d72-bf03-67387fab8ea1","Type":"ContainerStarted","Data":"b9df80bb8c38699bb7401d2bb05ad6d656013b186665f15fbbb7873f739bdc9c"}
Jul 09 18:11:56.211223 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:56.210839    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:11:56.227630 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:11:56.227584    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-jfwtb" podStartSLOduration=3.67835538 podStartE2EDuration="7.22756572s" totalImagesPullingTime="3.54921034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:56.22635688 +0000 UTC m=+478.573264797" watchObservedRunningTime="2026-07-09 18:11:56.22756572 +0000 UTC m=+478.574473635"
Jul 09 18:12:06.257641 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:06.257605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-5268b" event={"ID":"6df91432-09b3-48a0-b7f2-a6fc29b1f9ff","Type":"ContainerStarted","Data":"dc8448f105fdc580b4f977bb1f7d823e8e1bbb19ea6b03c0b32a4fc9ac4d76fc"}
Jul 09 18:12:06.258100 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:06.257713    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:12:06.275589 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:06.275541    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-5268b" podStartSLOduration=2.839813401 podStartE2EDuration="16.275525037s" totalImagesPullingTime="13.435711636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:12:06.273490234 +0000 UTC m=+488.620398151" watchObservedRunningTime="2026-07-09 18:12:06.275525037 +0000 UTC m=+488.622432955"
Jul 09 18:12:10.178131 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:10.178103    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-tfv95"
Jul 09 18:12:12.608217 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:12.608180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:17.217957 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:17.217925    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-jfwtb"
Jul 09 18:12:27.264834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:27.264775    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-5268b"
Jul 09 18:12:57.217969 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.217706    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.298545 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.298502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.333879 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.333850    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-92bdr"]
Jul 09 18:12:57.337324 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.337306    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.340509 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.340491    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 18:12:57.341051 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.341030    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-kmczv\""
Jul 09 18:12:57.341125 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.341071    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 18:12:57.341177 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.341122    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 18:12:57.351292 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.351269    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-92bdr"]
Jul 09 18:12:57.400380 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.400346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.483454 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.483362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgrd\" (UniqueName: \"kubernetes.io/projected/35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1-kube-api-access-xwgrd\") pod \"seaweedfs-64568bcd49-92bdr\" (UID: \"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1\") " pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.483454 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.483434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1-data\") pod \"seaweedfs-64568bcd49-92bdr\" (UID: \"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1\") " pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.584434 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.584401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgrd\" (UniqueName: \"kubernetes.io/projected/35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1-kube-api-access-xwgrd\") pod \"seaweedfs-64568bcd49-92bdr\" (UID: \"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1\") " pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.584605 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.584446    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1-data\") pod \"seaweedfs-64568bcd49-92bdr\" (UID: \"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1\") " pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.584807 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.584770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1-data\") pod \"seaweedfs-64568bcd49-92bdr\" (UID: \"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1\") " pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.592697 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.592665    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgrd\" (UniqueName: \"kubernetes.io/projected/35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1-kube-api-access-xwgrd\") pod \"seaweedfs-64568bcd49-92bdr\" (UID: \"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1\") " pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.611531 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.611506    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.648895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.648868    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:12:57.713372 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.713339    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.779011 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:57.778986    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-92bdr"]
Jul 09 18:12:57.781861 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:12:57.781835    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35afea2b_5da4_4ae8_9157_9cfb0cd9c9b1.slice/crio-8e8722cf4bc29afc54cc6ffcbfd306cfb774fa35863d2265f5b453599f549bd2 WatchSource:0}: Error finding container 8e8722cf4bc29afc54cc6ffcbfd306cfb774fa35863d2265f5b453599f549bd2: Status 404 returned error can't find the container with id 8e8722cf4bc29afc54cc6ffcbfd306cfb774fa35863d2265f5b453599f549bd2
Jul 09 18:12:58.174208 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:58.174120    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:58.275239 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:58.275200    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:58.339082 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:58.339046    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:58.511065 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:12:58.511018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-92bdr" event={"ID":"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1","Type":"ContainerStarted","Data":"8e8722cf4bc29afc54cc6ffcbfd306cfb774fa35863d2265f5b453599f549bd2"}
Jul 09 18:13:01.753553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:01.753523    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 18:13:02.531618 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:02.531575    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-92bdr" event={"ID":"35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1","Type":"ContainerStarted","Data":"24f783b581177d76e7b9c1f4de9f6bed0b1c4d4181f9d7ec6c344cff1e08ada2"}
Jul 09 18:13:02.531779 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:02.531665    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:13:02.547611 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:02.547560    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-92bdr" podStartSLOduration=1.579440041 podStartE2EDuration="5.547543771s" totalImagesPullingTime="3.96810373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:13:02.546730349 +0000 UTC m=+544.893638291" watchObservedRunningTime="2026-07-09 18:13:02.547543771 +0000 UTC m=+544.894451687"
Jul 09 18:13:08.538756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:08.538723    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-92bdr"
Jul 09 18:13:10.631610 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:10.631567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:10.706736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:10.706688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:12.803178 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:12.803139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:12.859548 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:12.859509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:14.553344 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:14.553304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:14.656092 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:14.656064    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:18.109305 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:18.109268    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:58.757004 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:58.756974    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:13:58.759283 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:13:58.759261    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:14:09.149750 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.149687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:09.210672 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.210628    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-hmptm"]
Jul 09 18:14:09.213136 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.213120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.215261 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.215237    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 09 18:14:09.215532 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.215515    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-nvdcj\""
Jul 09 18:14:09.228442 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.228422    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-hmptm"]
Jul 09 18:14:09.304983 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.304955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485647a0-d9fe-4a40-b666-3780ed978ce1-cert\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.305152 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.305011    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wmvs\" (UniqueName: \"kubernetes.io/projected/485647a0-d9fe-4a40-b666-3780ed978ce1-kube-api-access-9wmvs\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.405658 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.405580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmvs\" (UniqueName: \"kubernetes.io/projected/485647a0-d9fe-4a40-b666-3780ed978ce1-kube-api-access-9wmvs\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.405658 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.405644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485647a0-d9fe-4a40-b666-3780ed978ce1-cert\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.405902 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:14:09.405728    2567 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 09 18:14:09.405902 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:14:09.405777    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/485647a0-d9fe-4a40-b666-3780ed978ce1-cert podName:485647a0-d9fe-4a40-b666-3780ed978ce1 nodeName:}" failed. No retries permitted until 2026-07-09 18:14:09.905761776 +0000 UTC m=+612.252669670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/485647a0-d9fe-4a40-b666-3780ed978ce1-cert") pod "odh-model-controller-84b54b7f97-hmptm" (UID: "485647a0-d9fe-4a40-b666-3780ed978ce1") : secret "odh-model-controller-webhook-cert" not found
Jul 09 18:14:09.414624 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.414593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wmvs\" (UniqueName: \"kubernetes.io/projected/485647a0-d9fe-4a40-b666-3780ed978ce1-kube-api-access-9wmvs\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.911506 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.911477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485647a0-d9fe-4a40-b666-3780ed978ce1-cert\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:09.913959 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:09.913938    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485647a0-d9fe-4a40-b666-3780ed978ce1-cert\") pod \"odh-model-controller-84b54b7f97-hmptm\" (UID: \"485647a0-d9fe-4a40-b666-3780ed978ce1\") " pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:10.124205 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:10.124160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:10.258215 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:10.258191    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-hmptm"]
Jul 09 18:14:10.260432 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:14:10.260404    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485647a0_d9fe_4a40_b666_3780ed978ce1.slice/crio-dbc057b0830efec016471170ddbfba5f932e7ae76ce266d754acaf766a223e78 WatchSource:0}: Error finding container dbc057b0830efec016471170ddbfba5f932e7ae76ce266d754acaf766a223e78: Status 404 returned error can't find the container with id dbc057b0830efec016471170ddbfba5f932e7ae76ce266d754acaf766a223e78
Jul 09 18:14:10.261940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:10.261917    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:14:10.409903 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:10.409871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:10.851341 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:10.851306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-hmptm" event={"ID":"485647a0-d9fe-4a40-b666-3780ed978ce1","Type":"ContainerStarted","Data":"dbc057b0830efec016471170ddbfba5f932e7ae76ce266d754acaf766a223e78"}
Jul 09 18:14:13.871673 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:13.871639    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-hmptm" event={"ID":"485647a0-d9fe-4a40-b666-3780ed978ce1","Type":"ContainerStarted","Data":"e0a2b29e8b2b9d68e80964a9d9b55a205214d5bf4e1e54c0aca3b8b0f658475d"}
Jul 09 18:14:13.872069 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:13.871715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:13.891574 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:13.891521    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-hmptm" podStartSLOduration=1.975261004 podStartE2EDuration="4.891507965s" totalImagesPullingTime="2.916246961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:14:13.889053889 +0000 UTC m=+616.235961804" watchObservedRunningTime="2026-07-09 18:14:13.891507965 +0000 UTC m=+616.238415901"
Jul 09 18:14:24.879928 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:24.879897    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-hmptm"
Jul 09 18:14:25.741266 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.741230    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-mq4qw"]
Jul 09 18:14:25.743829 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.743808    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mq4qw"
Jul 09 18:14:25.752404 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.752384    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-mq4qw"]
Jul 09 18:14:25.858031 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.857997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757be998-2ce7-4d10-ac7c-b16df8cf728e-model-cache\") pod \"s3-init-mq4qw\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") " pod="kserve/s3-init-mq4qw"
Jul 09 18:14:25.858178 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.858048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhlxt\" (UniqueName: \"kubernetes.io/projected/757be998-2ce7-4d10-ac7c-b16df8cf728e-kube-api-access-zhlxt\") pod \"s3-init-mq4qw\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") " pod="kserve/s3-init-mq4qw"
Jul 09 18:14:25.958662 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.958632    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhlxt\" (UniqueName: \"kubernetes.io/projected/757be998-2ce7-4d10-ac7c-b16df8cf728e-kube-api-access-zhlxt\") pod \"s3-init-mq4qw\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") " pod="kserve/s3-init-mq4qw"
Jul 09 18:14:25.959072 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.958708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757be998-2ce7-4d10-ac7c-b16df8cf728e-model-cache\") pod \"s3-init-mq4qw\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") " pod="kserve/s3-init-mq4qw"
Jul 09 18:14:25.959120 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.959080    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757be998-2ce7-4d10-ac7c-b16df8cf728e-model-cache\") pod \"s3-init-mq4qw\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") " pod="kserve/s3-init-mq4qw"
Jul 09 18:14:25.967184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:25.967160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhlxt\" (UniqueName: \"kubernetes.io/projected/757be998-2ce7-4d10-ac7c-b16df8cf728e-kube-api-access-zhlxt\") pod \"s3-init-mq4qw\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") " pod="kserve/s3-init-mq4qw"
Jul 09 18:14:26.069305 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:26.069207    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mq4qw"
Jul 09 18:14:26.195816 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:14:26.195773    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod757be998_2ce7_4d10_ac7c_b16df8cf728e.slice/crio-147f8ecc3661f23790ad2285d72caf0110f0af71835457c12f4b1cb10e5987a4 WatchSource:0}: Error finding container 147f8ecc3661f23790ad2285d72caf0110f0af71835457c12f4b1cb10e5987a4: Status 404 returned error can't find the container with id 147f8ecc3661f23790ad2285d72caf0110f0af71835457c12f4b1cb10e5987a4
Jul 09 18:14:26.196467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:26.196442    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-mq4qw"]
Jul 09 18:14:26.942724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:26.942664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mq4qw" event={"ID":"757be998-2ce7-4d10-ac7c-b16df8cf728e","Type":"ContainerStarted","Data":"147f8ecc3661f23790ad2285d72caf0110f0af71835457c12f4b1cb10e5987a4"}
Jul 09 18:14:29.962107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:29.962072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mq4qw" event={"ID":"757be998-2ce7-4d10-ac7c-b16df8cf728e","Type":"ContainerStarted","Data":"ae6ce82d8abf7eb65153ef682503f548297693eefbc2af9bc9cce809c48c21d2"}
Jul 09 18:14:31.974716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:31.974682    2567 generic.go:356] "Generic (PLEG): container finished" podID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerID="ae6ce82d8abf7eb65153ef682503f548297693eefbc2af9bc9cce809c48c21d2" exitCode=0
Jul 09 18:14:31.975203 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:31.974721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mq4qw" event={"ID":"757be998-2ce7-4d10-ac7c-b16df8cf728e","Type":"ContainerDied","Data":"ae6ce82d8abf7eb65153ef682503f548297693eefbc2af9bc9cce809c48c21d2"}
Jul 09 18:14:33.512992 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:33.512937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:33.613639 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:33.613591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:33.713532 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:33.713489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:34.248593 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:34.248538    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:37.006239 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:37.006204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mq4qw" event={"ID":"757be998-2ce7-4d10-ac7c-b16df8cf728e","Type":"ContainerStarted","Data":"15b25b7303bc760276e1844aadc9c5e3b0a3a3569d87744a66b91310823a575e"}
Jul 09 18:14:37.023056 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:37.023006    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-mq4qw" podStartSLOduration=2.22468061 podStartE2EDuration="12.022990599s" totalImagesPullingTime="7.906694713s" totalInitContainerRuntime="1.891615276s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:14:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:14:37.021953626 +0000 UTC m=+639.368861567" watchObservedRunningTime="2026-07-09 18:14:37.022990599 +0000 UTC m=+639.369898527"
Jul 09 18:14:44.388504 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:44.388471    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5669459db9-2qvpl"]
Jul 09 18:14:44.497373 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:44.497338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:44.595386 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:44.595355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:44.654146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:14:44.654067    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:09.409004 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.408952    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5669459db9-2qvpl" podUID="9710a30f-56f3-4fd4-a91a-ed7484861b54" containerName="console" containerID="cri-o://13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28" gracePeriod=15
Jul 09 18:15:09.652110 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.652084    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:15:09.756682 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.756641    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht6pg\" (UniqueName: \"kubernetes.io/projected/9710a30f-56f3-4fd4-a91a-ed7484861b54-kube-api-access-ht6pg\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.756895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.756705    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-oauth-config\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.756895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.756729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-service-ca\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.756895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.756754    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-trusted-ca-bundle\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.756895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.756772    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-oauth-serving-cert\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.756895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.756853    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-config\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.757131 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757014    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-serving-cert\") pod \"9710a30f-56f3-4fd4-a91a-ed7484861b54\" (UID: \"9710a30f-56f3-4fd4-a91a-ed7484861b54\") "
Jul 09 18:15:09.757260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757221    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-service-ca" (OuterVolumeSpecName: "service-ca") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:15:09.757260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757236    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:15:09.757260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757254    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:15:09.757512 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757294    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-config" (OuterVolumeSpecName: "console-config") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:15:09.757512 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757379    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:09.757512 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757400    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-service-ca\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:09.757512 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757414    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-trusted-ca-bundle\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:09.757512 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.757427    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710a30f-56f3-4fd4-a91a-ed7484861b54-oauth-serving-cert\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:09.758959 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.758933    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:15:09.759091 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.759068    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:15:09.759162 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.759142    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9710a30f-56f3-4fd4-a91a-ed7484861b54-kube-api-access-ht6pg" (OuterVolumeSpecName: "kube-api-access-ht6pg") pod "9710a30f-56f3-4fd4-a91a-ed7484861b54" (UID: "9710a30f-56f3-4fd4-a91a-ed7484861b54"). InnerVolumeSpecName "kube-api-access-ht6pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:15:09.858747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.858698    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-serving-cert\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:09.858747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.858743    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht6pg\" (UniqueName: \"kubernetes.io/projected/9710a30f-56f3-4fd4-a91a-ed7484861b54-kube-api-access-ht6pg\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:09.858747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:09.858752    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710a30f-56f3-4fd4-a91a-ed7484861b54-console-oauth-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:10.173974 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.173888    2567 generic.go:356] "Generic (PLEG): container finished" podID="9710a30f-56f3-4fd4-a91a-ed7484861b54" containerID="13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28" exitCode=0
Jul 09 18:15:10.173974 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.173935    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5669459db9-2qvpl"
Jul 09 18:15:10.174146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.173971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5669459db9-2qvpl" event={"ID":"9710a30f-56f3-4fd4-a91a-ed7484861b54","Type":"ContainerDied","Data":"13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28"}
Jul 09 18:15:10.174146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.174027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5669459db9-2qvpl" event={"ID":"9710a30f-56f3-4fd4-a91a-ed7484861b54","Type":"ContainerDied","Data":"19a0fba3fb5545740c869347e6a3d7975dcbf481d0aba3a78a2fc51754b6d581"}
Jul 09 18:15:10.174146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.174050    2567 scope.go:117] "RemoveContainer" containerID="13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28"
Jul 09 18:15:10.183254 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.183238    2567 scope.go:117] "RemoveContainer" containerID="13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28"
Jul 09 18:15:10.183509 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:15:10.183489    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28\": container with ID starting with 13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28 not found: ID does not exist" containerID="13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28"
Jul 09 18:15:10.183595 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.183518    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28"} err="failed to get container status \"13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28\": rpc error: code = NotFound desc = could not find container \"13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28\": container with ID starting with 13bd77d7e21f6ecb5cec4f19bb2ecca3d4172b60d3c82abe7cd5027f12559d28 not found: ID does not exist"
Jul 09 18:15:10.195724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.195696    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5669459db9-2qvpl"]
Jul 09 18:15:10.199728 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:10.199707    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5669459db9-2qvpl"]
Jul 09 18:15:12.194474 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:12.194445    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9710a30f-56f3-4fd4-a91a-ed7484861b54" path="/var/lib/kubelet/pods/9710a30f-56f3-4fd4-a91a-ed7484861b54/volumes"
Jul 09 18:15:42.328081 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:42.328046    2567 generic.go:356] "Generic (PLEG): container finished" podID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerID="15b25b7303bc760276e1844aadc9c5e3b0a3a3569d87744a66b91310823a575e" exitCode=0
Jul 09 18:15:42.328523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:42.328116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mq4qw" event={"ID":"757be998-2ce7-4d10-ac7c-b16df8cf728e","Type":"ContainerDied","Data":"15b25b7303bc760276e1844aadc9c5e3b0a3a3569d87744a66b91310823a575e"}
Jul 09 18:15:43.465053 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.465031    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mq4qw"
Jul 09 18:15:43.542661 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.542625    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757be998-2ce7-4d10-ac7c-b16df8cf728e-model-cache\") pod \"757be998-2ce7-4d10-ac7c-b16df8cf728e\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") "
Jul 09 18:15:43.542889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.542715    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhlxt\" (UniqueName: \"kubernetes.io/projected/757be998-2ce7-4d10-ac7c-b16df8cf728e-kube-api-access-zhlxt\") pod \"757be998-2ce7-4d10-ac7c-b16df8cf728e\" (UID: \"757be998-2ce7-4d10-ac7c-b16df8cf728e\") "
Jul 09 18:15:43.544895 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.544855    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757be998-2ce7-4d10-ac7c-b16df8cf728e-kube-api-access-zhlxt" (OuterVolumeSpecName: "kube-api-access-zhlxt") pod "757be998-2ce7-4d10-ac7c-b16df8cf728e" (UID: "757be998-2ce7-4d10-ac7c-b16df8cf728e"). InnerVolumeSpecName "kube-api-access-zhlxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:15:43.561173 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.561151    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757be998-2ce7-4d10-ac7c-b16df8cf728e-model-cache" (OuterVolumeSpecName: "model-cache") pod "757be998-2ce7-4d10-ac7c-b16df8cf728e" (UID: "757be998-2ce7-4d10-ac7c-b16df8cf728e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:15:43.644207 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.644134    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757be998-2ce7-4d10-ac7c-b16df8cf728e-model-cache\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:43.644207 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:43.644170    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhlxt\" (UniqueName: \"kubernetes.io/projected/757be998-2ce7-4d10-ac7c-b16df8cf728e-kube-api-access-zhlxt\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:15:44.339871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:44.339835    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mq4qw" event={"ID":"757be998-2ce7-4d10-ac7c-b16df8cf728e","Type":"ContainerDied","Data":"147f8ecc3661f23790ad2285d72caf0110f0af71835457c12f4b1cb10e5987a4"}
Jul 09 18:15:44.339871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:44.339874    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="147f8ecc3661f23790ad2285d72caf0110f0af71835457c12f4b1cb10e5987a4"
Jul 09 18:15:44.339871 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:44.339874    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mq4qw"
Jul 09 18:15:44.710769 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:44.710733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:44.822151 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:44.822119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:56.480525 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.480425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:56.532701 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.532673    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"]
Jul 09 18:15:56.533082 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533069    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerName="s3-init"
Jul 09 18:15:56.533148 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533083    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerName="s3-init"
Jul 09 18:15:56.533148 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533096    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9710a30f-56f3-4fd4-a91a-ed7484861b54" containerName="console"
Jul 09 18:15:56.533148 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533102    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9710a30f-56f3-4fd4-a91a-ed7484861b54" containerName="console"
Jul 09 18:15:56.533148 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533112    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerName="download-hf-model"
Jul 09 18:15:56.533148 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533117    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerName="download-hf-model"
Jul 09 18:15:56.533399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533181    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9710a30f-56f3-4fd4-a91a-ed7484861b54" containerName="console"
Jul 09 18:15:56.533399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.533190    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="757be998-2ce7-4d10-ac7c-b16df8cf728e" containerName="s3-init"
Jul 09 18:15:56.536244 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.536226    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.538870 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.538845    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6s7vj\""
Jul 09 18:15:56.538996 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.538868    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:15:56.538996 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.538845    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Jul 09 18:15:56.539621 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.539601    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:15:56.539731 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.539621    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:15:56.539731 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.539659    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 18:15:56.539731 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.539658    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Jul 09 18:15:56.547899 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.547880    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"]
Jul 09 18:15:56.657119 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.657080    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b6dv\" (UniqueName: \"kubernetes.io/projected/9887975e-9a70-408a-958b-450bf8eed2fd-kube-api-access-4b6dv\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.657289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.657196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.657289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.657244    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9887975e-9a70-408a-958b-450bf8eed2fd-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.657427 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.657323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9887975e-9a70-408a-958b-450bf8eed2fd-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.657427 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.657359    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.669998 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.669945    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:56.704717 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.704680    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"]
Jul 09 18:15:56.708435 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.708416    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.710742 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.710716    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-db4c0-kube-rbac-proxy-sar-config\""
Jul 09 18:15:56.710873 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.710854    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-db4c0-predictor-serving-cert\""
Jul 09 18:15:56.721213 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.721189    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"]
Jul 09 18:15:56.758419 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.758419 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758396    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.758636 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9887975e-9a70-408a-958b-450bf8eed2fd-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.758636 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.758636 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758613    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9887975e-9a70-408a-958b-450bf8eed2fd-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.758826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.758826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b6dv\" (UniqueName: \"kubernetes.io/projected/9887975e-9a70-408a-958b-450bf8eed2fd-kube-api-access-4b6dv\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.758826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758736    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv8xf\" (UniqueName: \"kubernetes.io/projected/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-api-access-dv8xf\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.758999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.758984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9887975e-9a70-408a-958b-450bf8eed2fd-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.759200 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.759175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.759744 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.759548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.761430 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.761405    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9887975e-9a70-408a-958b-450bf8eed2fd-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.769740 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.769719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b6dv\" (UniqueName: \"kubernetes.io/projected/9887975e-9a70-408a-958b-450bf8eed2fd-kube-api-access-4b6dv\") pod \"isvc-sklearn-graph-1-predictor-7595977785-zw5wr\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.778476 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.778445    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:56.812813 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.812769    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"]
Jul 09 18:15:56.816393 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.816376    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.818443 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.818421    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Jul 09 18:15:56.818557 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.818423    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Jul 09 18:15:56.827614 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.827589    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"]
Jul 09 18:15:56.847948 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.847921    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:15:56.860044 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.859978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.860044 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.860234 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.860234 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.860234 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/341bbd2a-973f-442d-8458-c3c633781933-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.860234 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/341bbd2a-973f-442d-8458-c3c633781933-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.860451 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860268    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv8xf\" (UniqueName: \"kubernetes.io/projected/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-api-access-dv8xf\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.860451 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zr8j\" (UniqueName: \"kubernetes.io/projected/341bbd2a-973f-442d-8458-c3c633781933-kube-api-access-9zr8j\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.860552 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:15:56.860451    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-serving-cert: secret "success-200-isvc-db4c0-predictor-serving-cert" not found
Jul 09 18:15:56.860552 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:15:56.860519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls podName:6d754eab-2603-443f-ae67-3d877e6cfc2f nodeName:}" failed. No retries permitted until 2026-07-09 18:15:57.360499598 +0000 UTC m=+719.707407493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls") pod "success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" (UID: "6d754eab-2603-443f-ae67-3d877e6cfc2f") : secret "success-200-isvc-db4c0-predictor-serving-cert" not found
Jul 09 18:15:56.860742 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.860718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.870950 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.870918    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv8xf\" (UniqueName: \"kubernetes.io/projected/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-api-access-dv8xf\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:56.878546 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.878480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:56.949123 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.949059    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:56.961861 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.961107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zr8j\" (UniqueName: \"kubernetes.io/projected/341bbd2a-973f-442d-8458-c3c633781933-kube-api-access-9zr8j\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.961861 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.961180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.961861 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.961233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.961861 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.961290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/341bbd2a-973f-442d-8458-c3c633781933-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.961861 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.961332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/341bbd2a-973f-442d-8458-c3c633781933-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.962195 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.962117    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.962412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.962389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.962480 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.962452    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/341bbd2a-973f-442d-8458-c3c633781933-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.965080 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.965032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/341bbd2a-973f-442d-8458-c3c633781933-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:56.973429 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:56.973380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zr8j\" (UniqueName: \"kubernetes.io/projected/341bbd2a-973f-442d-8458-c3c633781933-kube-api-access-9zr8j\") pod \"isvc-xgboost-graph-predictor-555fc69d56-zqrtr\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:57.014014 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.013968    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"]
Jul 09 18:15:57.016152 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:15:57.016114    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9887975e_9a70_408a_958b_450bf8eed2fd.slice/crio-a1e400670506c1173ab7dd246e86dbf716d69c8452e00792474c53e55756b60c WatchSource:0}: Error finding container a1e400670506c1173ab7dd246e86dbf716d69c8452e00792474c53e55756b60c: Status 404 returned error can't find the container with id a1e400670506c1173ab7dd246e86dbf716d69c8452e00792474c53e55756b60c
Jul 09 18:15:57.019190 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.019145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:57.101015 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.100986    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:57.131649 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.131621    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:15:57.283562 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.283529    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"]
Jul 09 18:15:57.285343 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:15:57.285309    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341bbd2a_973f_442d_8458_c3c633781933.slice/crio-28c793da93c2c0a16f686b98664301a82d0efb5a9c6ff9551d7c2d4eed3c75e7 WatchSource:0}: Error finding container 28c793da93c2c0a16f686b98664301a82d0efb5a9c6ff9551d7c2d4eed3c75e7: Status 404 returned error can't find the container with id 28c793da93c2c0a16f686b98664301a82d0efb5a9c6ff9551d7c2d4eed3c75e7
Jul 09 18:15:57.366154 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.366119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:57.369081 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.369053    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls\") pod \"success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") " pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:57.412819 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.412749    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerStarted","Data":"c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827"}
Jul 09 18:15:57.413022 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.412853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerStarted","Data":"28c793da93c2c0a16f686b98664301a82d0efb5a9c6ff9551d7c2d4eed3c75e7"}
Jul 09 18:15:57.414727 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.414687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerStarted","Data":"0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01"}
Jul 09 18:15:57.414882 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.414758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerStarted","Data":"a1e400670506c1173ab7dd246e86dbf716d69c8452e00792474c53e55756b60c"}
Jul 09 18:15:57.441399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.441353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:57.620802 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.620702    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:15:57.757756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:57.757731    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"]
Jul 09 18:15:57.759393 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:15:57.759363    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d754eab_2603_443f_ae67_3d877e6cfc2f.slice/crio-eb39d1cfc46b5f60f091b9e99fd7d3d4c0d5ca4c7505f8d326a2f3e1c3f5721f WatchSource:0}: Error finding container eb39d1cfc46b5f60f091b9e99fd7d3d4c0d5ca4c7505f8d326a2f3e1c3f5721f: Status 404 returned error can't find the container with id eb39d1cfc46b5f60f091b9e99fd7d3d4c0d5ca4c7505f8d326a2f3e1c3f5721f
Jul 09 18:15:58.147385 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:58.147359    2567 scope.go:117] "RemoveContainer" containerID="b61de603f146176a1aecb4ffe6677c28eacba9431a5861d1fabf7ee83e3f6681"
Jul 09 18:15:58.430573 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:15:58.430488    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" event={"ID":"6d754eab-2603-443f-ae67-3d877e6cfc2f","Type":"ContainerStarted","Data":"eb39d1cfc46b5f60f091b9e99fd7d3d4c0d5ca4c7505f8d326a2f3e1c3f5721f"}
Jul 09 18:16:03.464570 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:03.464535    2567 generic.go:356] "Generic (PLEG): container finished" podID="341bbd2a-973f-442d-8458-c3c633781933" containerID="c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827" exitCode=0
Jul 09 18:16:03.465090 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:03.464625    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerDied","Data":"c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827"}
Jul 09 18:16:03.466631 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:03.466599    2567 generic.go:356] "Generic (PLEG): container finished" podID="9887975e-9a70-408a-958b-450bf8eed2fd" containerID="0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01" exitCode=0
Jul 09 18:16:03.466746 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:03.466637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerDied","Data":"0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01"}
Jul 09 18:16:11.520510 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:11.520446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" event={"ID":"6d754eab-2603-443f-ae67-3d877e6cfc2f","Type":"ContainerStarted","Data":"259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55"}
Jul 09 18:16:18.570727 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.570688    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" event={"ID":"6d754eab-2603-443f-ae67-3d877e6cfc2f","Type":"ContainerStarted","Data":"4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085"}
Jul 09 18:16:18.571260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.570886    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:16:18.571260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.571061    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:16:18.572469 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.572436    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:16:18.573574 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.573551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerStarted","Data":"30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4"}
Jul 09 18:16:18.573688 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.573606    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerStarted","Data":"f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b"}
Jul 09 18:16:18.573837 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.573811    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:16:18.590500 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.590420    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podStartSLOduration=2.606964327 podStartE2EDuration="22.590403329s" totalImagesPullingTime="19.983439002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:15:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:16:18.587520271 +0000 UTC m=+740.934428189" watchObservedRunningTime="2026-07-09 18:16:18.590403329 +0000 UTC m=+740.937311246"
Jul 09 18:16:18.606820 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:18.606747    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podStartSLOduration=4.405436807 podStartE2EDuration="22.606726174s" totalImagesPullingTime="14.368128595s" totalInitContainerRuntime="3.833160772s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:16:18.60475887 +0000 UTC m=+740.951666787" watchObservedRunningTime="2026-07-09 18:16:18.606726174 +0000 UTC m=+740.953634091"
Jul 09 18:16:19.579286 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:19.579234    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:16:19.579758 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:19.579330    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:16:19.580743 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:19.580718    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:16:20.584940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:20.584900    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:16:24.585249 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:24.584477    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:16:24.585708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:24.585337    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:16:25.590252 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:25.590217    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:16:25.590864 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:25.590828    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:16:33.662865 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:33.662819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerStarted","Data":"c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa"}
Jul 09 18:16:33.663322 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:33.662903    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerStarted","Data":"3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852"}
Jul 09 18:16:33.663322 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:33.663216    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:16:33.663447 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:33.663361    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:16:33.664726 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:33.664698    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:16:33.682760 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:33.682715    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podStartSLOduration=4.499407773 podStartE2EDuration="37.682700328s" totalImagesPullingTime="29.473926596s" totalInitContainerRuntime="3.709365959s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:16:33.680175392 +0000 UTC m=+756.027083309" watchObservedRunningTime="2026-07-09 18:16:33.682700328 +0000 UTC m=+756.029608244"
Jul 09 18:16:34.585465 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:34.585423    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:16:34.668549 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:34.668509    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:16:35.591756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:35.591711    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:16:37.592225 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:37.592169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:37.653112 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:37.653076    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:39.673156 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:39.673126    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:16:39.673715 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:39.673684    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:16:44.585164 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:44.585111    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:16:45.591940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:45.591900    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:16:49.673831 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:49.673767    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:16:54.585835 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:54.585775    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:16:55.590911 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:55.590871    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:16:59.674138 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:16:59.674093    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:17:04.585122 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:04.585076    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 09 18:17:05.591077 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:05.591026    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:17:09.674125 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:09.674081    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:17:14.585689 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:14.585659    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:17:15.591329 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:15.591286    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:17:19.674295 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:19.674254    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:17:25.590914 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:25.590815    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:17:26.517231 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:26.517193    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:26.894073 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:26.893983    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:29.674099 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:29.674056    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:17:33.189869 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:33.189827    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 09 18:17:39.674473 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:39.674441    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:17:40.686855 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.686816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:40.812462 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.812424    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"]
Jul 09 18:17:40.812885 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.812701    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container" containerID="cri-o://259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55" gracePeriod=30
Jul 09 18:17:40.812885 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.812740    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kube-rbac-proxy" containerID="cri-o://4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085" gracePeriod=30
Jul 09 18:17:40.832701 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.832672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:40.870540 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.870503    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"]
Jul 09 18:17:40.874171 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.874145    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:40.876484 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.876456    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2a7bf-kube-rbac-proxy-sar-config\""
Jul 09 18:17:40.876639 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.876461    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2a7bf-predictor-serving-cert\""
Jul 09 18:17:40.887941 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.887913    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"]
Jul 09 18:17:40.910439 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.910405    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:40.986644 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:40.986598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:41.005736 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.005704    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerID="4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085" exitCode=2
Jul 09 18:17:41.005924 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.005772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" event={"ID":"6d754eab-2603-443f-ae67-3d877e6cfc2f","Type":"ContainerDied","Data":"4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085"}
Jul 09 18:17:41.038373 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.038346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.038500 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.038392    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/346318b7-30ec-4943-90dd-bf4f8583f156-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.038555 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.038497    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb2dw\" (UniqueName: \"kubernetes.io/projected/346318b7-30ec-4943-90dd-bf4f8583f156-kube-api-access-nb2dw\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.139071 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.139032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/346318b7-30ec-4943-90dd-bf4f8583f156-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.139246 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.139098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb2dw\" (UniqueName: \"kubernetes.io/projected/346318b7-30ec-4943-90dd-bf4f8583f156-kube-api-access-nb2dw\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.139246 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.139185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.139346 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:17:41.139306    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-serving-cert: secret "success-200-isvc-2a7bf-predictor-serving-cert" not found
Jul 09 18:17:41.139385 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:17:41.139364    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls podName:346318b7-30ec-4943-90dd-bf4f8583f156 nodeName:}" failed. No retries permitted until 2026-07-09 18:17:41.639344847 +0000 UTC m=+823.986252744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls") pod "success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" (UID: "346318b7-30ec-4943-90dd-bf4f8583f156") : secret "success-200-isvc-2a7bf-predictor-serving-cert" not found
Jul 09 18:17:41.139740 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.139720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/346318b7-30ec-4943-90dd-bf4f8583f156-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.147724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.147699    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb2dw\" (UniqueName: \"kubernetes.io/projected/346318b7-30ec-4943-90dd-bf4f8583f156-kube-api-access-nb2dw\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.178295 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.178257    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:41.297999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.297909    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:41.643725 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.643633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.646058 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.646038    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls\") pod \"success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") " pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.787904 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.787869    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:41.923424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:41.923399    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"]
Jul 09 18:17:41.925748 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:17:41.925719    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346318b7_30ec_4943_90dd_bf4f8583f156.slice/crio-8e645d77978072b20895d575d54befd732847873a9f6552529883246fbf4dd6f WatchSource:0}: Error finding container 8e645d77978072b20895d575d54befd732847873a9f6552529883246fbf4dd6f: Status 404 returned error can't find the container with id 8e645d77978072b20895d575d54befd732847873a9f6552529883246fbf4dd6f
Jul 09 18:17:42.013580 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:42.013553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" event={"ID":"346318b7-30ec-4943-90dd-bf4f8583f156","Type":"ContainerStarted","Data":"b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52"}
Jul 09 18:17:42.013672 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:42.013613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" event={"ID":"346318b7-30ec-4943-90dd-bf4f8583f156","Type":"ContainerStarted","Data":"8e645d77978072b20895d575d54befd732847873a9f6552529883246fbf4dd6f"}
Jul 09 18:17:43.020746 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.020711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" event={"ID":"346318b7-30ec-4943-90dd-bf4f8583f156","Type":"ContainerStarted","Data":"f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f"}
Jul 09 18:17:43.021185 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.020890    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:43.021185 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.020954    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:43.022342 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.022316    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 09 18:17:43.039434 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.039377    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podStartSLOduration=3.039362921 podStartE2EDuration="3.039362921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:17:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:17:43.037177068 +0000 UTC m=+825.384084995" watchObservedRunningTime="2026-07-09 18:17:43.039362921 +0000 UTC m=+825.386270836"
Jul 09 18:17:43.191008 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.190979    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:17:43.962444 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:43.962414    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:17:44.027369 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.027341    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerID="259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55" exitCode=0
Jul 09 18:17:44.027868 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.027392    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"
Jul 09 18:17:44.027868 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.027429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" event={"ID":"6d754eab-2603-443f-ae67-3d877e6cfc2f","Type":"ContainerDied","Data":"259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55"}
Jul 09 18:17:44.027868 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.027539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5" event={"ID":"6d754eab-2603-443f-ae67-3d877e6cfc2f","Type":"ContainerDied","Data":"eb39d1cfc46b5f60f091b9e99fd7d3d4c0d5ca4c7505f8d326a2f3e1c3f5721f"}
Jul 09 18:17:44.027868 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.027576    2567 scope.go:117] "RemoveContainer" containerID="4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085"
Jul 09 18:17:44.028095 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.028029    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 09 18:17:44.036042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.036024    2567 scope.go:117] "RemoveContainer" containerID="259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55"
Jul 09 18:17:44.043421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.043404    2567 scope.go:117] "RemoveContainer" containerID="4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085"
Jul 09 18:17:44.043656 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:17:44.043637    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085\": container with ID starting with 4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085 not found: ID does not exist" containerID="4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085"
Jul 09 18:17:44.043702 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.043667    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085"} err="failed to get container status \"4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085\": rpc error: code = NotFound desc = could not find container \"4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085\": container with ID starting with 4f2a993c616ab696a98ff8e4cbbb2377069a91d80ea6097d09bcb1b4c117b085 not found: ID does not exist"
Jul 09 18:17:44.043702 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.043689    2567 scope.go:117] "RemoveContainer" containerID="259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55"
Jul 09 18:17:44.043960 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:17:44.043942    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55\": container with ID starting with 259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55 not found: ID does not exist" containerID="259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55"
Jul 09 18:17:44.044009 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.043964    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55"} err="failed to get container status \"259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55\": rpc error: code = NotFound desc = could not find container \"259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55\": container with ID starting with 259e54082f1372c8aca7c7246d330af4e164d123a5445461f8c3b43f1354ca55 not found: ID does not exist"
Jul 09 18:17:44.063426 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.063395    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls\") pod \"6d754eab-2603-443f-ae67-3d877e6cfc2f\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") "
Jul 09 18:17:44.063592 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.063577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv8xf\" (UniqueName: \"kubernetes.io/projected/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-api-access-dv8xf\") pod \"6d754eab-2603-443f-ae67-3d877e6cfc2f\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") "
Jul 09 18:17:44.063636 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.063609    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-rbac-proxy-sar-config\") pod \"6d754eab-2603-443f-ae67-3d877e6cfc2f\" (UID: \"6d754eab-2603-443f-ae67-3d877e6cfc2f\") "
Jul 09 18:17:44.063967 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.063943    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6d754eab-2603-443f-ae67-3d877e6cfc2f" (UID: "6d754eab-2603-443f-ae67-3d877e6cfc2f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:17:44.065396 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.065368    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6d754eab-2603-443f-ae67-3d877e6cfc2f" (UID: "6d754eab-2603-443f-ae67-3d877e6cfc2f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:17:44.065482 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.065467    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-api-access-dv8xf" (OuterVolumeSpecName: "kube-api-access-dv8xf") pod "6d754eab-2603-443f-ae67-3d877e6cfc2f" (UID: "6d754eab-2603-443f-ae67-3d877e6cfc2f"). InnerVolumeSpecName "kube-api-access-dv8xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:17:44.165160 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.165124    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d754eab-2603-443f-ae67-3d877e6cfc2f-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:17:44.165160 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.165155    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv8xf\" (UniqueName: \"kubernetes.io/projected/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-api-access-dv8xf\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:17:44.165160 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.165166    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d754eab-2603-443f-ae67-3d877e6cfc2f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:17:44.343431 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.343356    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"]
Jul 09 18:17:44.348840 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:44.348813    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-db4c0-predictor-845dccc9d9-9jqw5"]
Jul 09 18:17:46.194455 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:46.194422    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" path="/var/lib/kubelet/pods/6d754eab-2603-443f-ae67-3d877e6cfc2f/volumes"
Jul 09 18:17:49.033723 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:49.033692    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:17:49.034234 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:49.034208    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 09 18:17:59.034495 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:17:59.034454    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 09 18:18:06.506724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:06.506680    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:07.054739 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:07.054704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:09.034951 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:09.034911    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 09 18:18:16.607578 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.607541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:16.677354 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.677320    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"]
Jul 09 18:18:16.677581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.677547    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" containerID="cri-o://3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852" gracePeriod=30
Jul 09 18:18:16.677716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.677583    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kube-rbac-proxy" containerID="cri-o://c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa" gracePeriod=30
Jul 09 18:18:16.709325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.709230    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:16.796450 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.796421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"]
Jul 09 18:18:16.796709 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.796681    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container" containerID="cri-o://f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b" gracePeriod=30
Jul 09 18:18:16.797166 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.796817    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kube-rbac-proxy" containerID="cri-o://30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4" gracePeriod=30
Jul 09 18:18:16.799309 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.799289    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"]
Jul 09 18:18:16.799707 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.799681    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:16.799961 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.799878    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kube-rbac-proxy"
Jul 09 18:18:16.799961 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.799901    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kube-rbac-proxy"
Jul 09 18:18:16.799961 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.799920    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container"
Jul 09 18:18:16.799961 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.799929    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container"
Jul 09 18:18:16.800221 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.800031    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kserve-container"
Jul 09 18:18:16.800221 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.800050    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d754eab-2603-443f-ae67-3d877e6cfc2f" containerName="kube-rbac-proxy"
Jul 09 18:18:16.802523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.802399    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.804977 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.804952    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e8119-predictor-serving-cert\""
Jul 09 18:18:16.805081 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.804998    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e8119-kube-rbac-proxy-sar-config\""
Jul 09 18:18:16.813600 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.813580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"]
Jul 09 18:18:16.860879 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.860813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.860879 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.860855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7125cef7-4588-486c-9464-695fffa08ba8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.861038 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.860921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z55xp\" (UniqueName: \"kubernetes.io/projected/7125cef7-4588-486c-9464-695fffa08ba8-kube-api-access-z55xp\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.869494 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.869442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:16.941317 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.941286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:16.961835 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.961776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.961975 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.961858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7125cef7-4588-486c-9464-695fffa08ba8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.961975 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:16.961922    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-e8119-predictor-serving-cert: secret "success-200-isvc-e8119-predictor-serving-cert" not found
Jul 09 18:18:16.961975 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.961942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z55xp\" (UniqueName: \"kubernetes.io/projected/7125cef7-4588-486c-9464-695fffa08ba8-kube-api-access-z55xp\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.962119 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:16.961994    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls podName:7125cef7-4588-486c-9464-695fffa08ba8 nodeName:}" failed. No retries permitted until 2026-07-09 18:18:17.461973659 +0000 UTC m=+859.808881558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls") pod "success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" (UID: "7125cef7-4588-486c-9464-695fffa08ba8") : secret "success-200-isvc-e8119-predictor-serving-cert" not found
Jul 09 18:18:16.962559 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.962537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7125cef7-4588-486c-9464-695fffa08ba8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:16.980486 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:16.980462    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z55xp\" (UniqueName: \"kubernetes.io/projected/7125cef7-4588-486c-9464-695fffa08ba8-kube-api-access-z55xp\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:17.196164 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.196130    2567 generic.go:356] "Generic (PLEG): container finished" podID="341bbd2a-973f-442d-8458-c3c633781933" containerID="c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa" exitCode=2
Jul 09 18:18:17.196364 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.196199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerDied","Data":"c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa"}
Jul 09 18:18:17.198310 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.198287    2567 generic.go:356] "Generic (PLEG): container finished" podID="9887975e-9a70-408a-958b-450bf8eed2fd" containerID="30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4" exitCode=2
Jul 09 18:18:17.198446 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.198357    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerDied","Data":"30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4"}
Jul 09 18:18:17.467968 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.467875    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:17.470281 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.470253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls\") pod \"success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") " pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:17.723304 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.723210    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:17.859187 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.859008    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:17.859828 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.859802    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"]
Jul 09 18:18:17.861654 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:18:17.861620    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7125cef7_4588_486c_9464_695fffa08ba8.slice/crio-25e92fb1fad1f77aa8e4672e10e6d525de352a255a39460247e3fdd8a2e756ce WatchSource:0}: Error finding container 25e92fb1fad1f77aa8e4672e10e6d525de352a255a39460247e3fdd8a2e756ce: Status 404 returned error can't find the container with id 25e92fb1fad1f77aa8e4672e10e6d525de352a255a39460247e3fdd8a2e756ce
Jul 09 18:18:17.958396 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:17.958368    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:18.204639 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.204599    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" event={"ID":"7125cef7-4588-486c-9464-695fffa08ba8","Type":"ContainerStarted","Data":"d2ffc18030f7dbf029a327ec5c1028aadfd7cf6e6d05e11576ba9eef07fee3ee"}
Jul 09 18:18:18.204852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.204673    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:18.204852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.204704    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" event={"ID":"7125cef7-4588-486c-9464-695fffa08ba8","Type":"ContainerStarted","Data":"55c2a1d5302f64470c87296738a8970d28473e602d56337eaa91b75dcb68098f"}
Jul 09 18:18:18.204852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.204735    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:18.204852 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.204760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" event={"ID":"7125cef7-4588-486c-9464-695fffa08ba8","Type":"ContainerStarted","Data":"25e92fb1fad1f77aa8e4672e10e6d525de352a255a39460247e3fdd8a2e756ce"}
Jul 09 18:18:18.206222 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.206193    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:18:18.224069 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:18.224017    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podStartSLOduration=2.224001081 podStartE2EDuration="2.224001081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:18:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:18:18.221335897 +0000 UTC m=+860.568243814" watchObservedRunningTime="2026-07-09 18:18:18.224001081 +0000 UTC m=+860.570909009"
Jul 09 18:18:19.034272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:19.034232    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 09 18:18:19.209436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:19.209399    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:18:19.668938 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:19.668887    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Jul 09 18:18:19.674064 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:19.674037    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 09 18:18:20.585683 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.585636    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 09 18:18:20.640543 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.640481    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:18:20.697919 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.697888    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-cabundle-cert\") pod \"341bbd2a-973f-442d-8458-c3c633781933\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") "
Jul 09 18:18:20.698107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.697936    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/341bbd2a-973f-442d-8458-c3c633781933-kserve-provision-location\") pod \"341bbd2a-973f-442d-8458-c3c633781933\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") "
Jul 09 18:18:20.698107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.698009    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zr8j\" (UniqueName: \"kubernetes.io/projected/341bbd2a-973f-442d-8458-c3c633781933-kube-api-access-9zr8j\") pod \"341bbd2a-973f-442d-8458-c3c633781933\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") "
Jul 09 18:18:20.698107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.698038    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-kube-rbac-proxy-sar-config\") pod \"341bbd2a-973f-442d-8458-c3c633781933\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") "
Jul 09 18:18:20.698107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.698065    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/341bbd2a-973f-442d-8458-c3c633781933-proxy-tls\") pod \"341bbd2a-973f-442d-8458-c3c633781933\" (UID: \"341bbd2a-973f-442d-8458-c3c633781933\") "
Jul 09 18:18:20.698353 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.698325    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341bbd2a-973f-442d-8458-c3c633781933-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "341bbd2a-973f-442d-8458-c3c633781933" (UID: "341bbd2a-973f-442d-8458-c3c633781933"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:18:20.698428 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.698403    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "341bbd2a-973f-442d-8458-c3c633781933" (UID: "341bbd2a-973f-442d-8458-c3c633781933"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:18:20.698467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.698424    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "341bbd2a-973f-442d-8458-c3c633781933" (UID: "341bbd2a-973f-442d-8458-c3c633781933"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:18:20.700169 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.700141    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/341bbd2a-973f-442d-8458-c3c633781933-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "341bbd2a-973f-442d-8458-c3c633781933" (UID: "341bbd2a-973f-442d-8458-c3c633781933"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:18:20.700289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.700186    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341bbd2a-973f-442d-8458-c3c633781933-kube-api-access-9zr8j" (OuterVolumeSpecName: "kube-api-access-9zr8j") pod "341bbd2a-973f-442d-8458-c3c633781933" (UID: "341bbd2a-973f-442d-8458-c3c633781933"). InnerVolumeSpecName "kube-api-access-9zr8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:18:20.798833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.798774    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zr8j\" (UniqueName: \"kubernetes.io/projected/341bbd2a-973f-442d-8458-c3c633781933-kube-api-access-9zr8j\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:20.798833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.798827    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:20.798833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.798837    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/341bbd2a-973f-442d-8458-c3c633781933-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:20.798833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.798846    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/341bbd2a-973f-442d-8458-c3c633781933-cabundle-cert\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:20.799088 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:20.798855    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/341bbd2a-973f-442d-8458-c3c633781933-kserve-provision-location\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:21.221088 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.221053    2567 generic.go:356] "Generic (PLEG): container finished" podID="341bbd2a-973f-442d-8458-c3c633781933" containerID="3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852" exitCode=0
Jul 09 18:18:21.221290 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.221127    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"
Jul 09 18:18:21.221290 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.221128    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerDied","Data":"3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852"}
Jul 09 18:18:21.228264 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.221549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr" event={"ID":"341bbd2a-973f-442d-8458-c3c633781933","Type":"ContainerDied","Data":"28c793da93c2c0a16f686b98664301a82d0efb5a9c6ff9551d7c2d4eed3c75e7"}
Jul 09 18:18:21.228264 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.221636    2567 scope.go:117] "RemoveContainer" containerID="c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa"
Jul 09 18:18:21.237865 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.237830    2567 scope.go:117] "RemoveContainer" containerID="3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852"
Jul 09 18:18:21.246020 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.246004    2567 scope.go:117] "RemoveContainer" containerID="c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827"
Jul 09 18:18:21.247535 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.247511    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"]
Jul 09 18:18:21.251714 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.251691    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-zqrtr"]
Jul 09 18:18:21.254302 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.254283    2567 scope.go:117] "RemoveContainer" containerID="c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa"
Jul 09 18:18:21.254544 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:21.254525    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa\": container with ID starting with c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa not found: ID does not exist" containerID="c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa"
Jul 09 18:18:21.254594 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.254552    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa"} err="failed to get container status \"c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa\": rpc error: code = NotFound desc = could not find container \"c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa\": container with ID starting with c4a929787c011cdbe829b8da3d77aa44755c899862662d5c12f48ebdbcf51bfa not found: ID does not exist"
Jul 09 18:18:21.254594 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.254569    2567 scope.go:117] "RemoveContainer" containerID="3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852"
Jul 09 18:18:21.254829 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:21.254806    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852\": container with ID starting with 3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852 not found: ID does not exist" containerID="3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852"
Jul 09 18:18:21.254934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.254832    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852"} err="failed to get container status \"3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852\": rpc error: code = NotFound desc = could not find container \"3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852\": container with ID starting with 3a469eceb1331391573b2376ee0b4ae1a03764583ddf1096cba85345046ac852 not found: ID does not exist"
Jul 09 18:18:21.254934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.254846    2567 scope.go:117] "RemoveContainer" containerID="c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827"
Jul 09 18:18:21.255072 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:21.255056    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827\": container with ID starting with c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827 not found: ID does not exist" containerID="c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827"
Jul 09 18:18:21.255113 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.255076    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827"} err="failed to get container status \"c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827\": rpc error: code = NotFound desc = could not find container \"c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827\": container with ID starting with c30c3b3fc1d1c5b40455a6aa2342a2a8cee1a6b95b6d596942affcbc65d57827 not found: ID does not exist"
Jul 09 18:18:21.849880 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.849855    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:18:21.908505 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.908470    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b6dv\" (UniqueName: \"kubernetes.io/projected/9887975e-9a70-408a-958b-450bf8eed2fd-kube-api-access-4b6dv\") pod \"9887975e-9a70-408a-958b-450bf8eed2fd\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") "
Jul 09 18:18:21.908713 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.908518    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-kube-rbac-proxy-sar-config\") pod \"9887975e-9a70-408a-958b-450bf8eed2fd\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") "
Jul 09 18:18:21.908713 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.908577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9887975e-9a70-408a-958b-450bf8eed2fd-proxy-tls\") pod \"9887975e-9a70-408a-958b-450bf8eed2fd\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") "
Jul 09 18:18:21.908713 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.908661    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-cabundle-cert\") pod \"9887975e-9a70-408a-958b-450bf8eed2fd\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") "
Jul 09 18:18:21.908713 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.908697    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9887975e-9a70-408a-958b-450bf8eed2fd-kserve-provision-location\") pod \"9887975e-9a70-408a-958b-450bf8eed2fd\" (UID: \"9887975e-9a70-408a-958b-450bf8eed2fd\") "
Jul 09 18:18:21.909053 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.909020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9887975e-9a70-408a-958b-450bf8eed2fd" (UID: "9887975e-9a70-408a-958b-450bf8eed2fd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:18:21.909118 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.909063    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9887975e-9a70-408a-958b-450bf8eed2fd" (UID: "9887975e-9a70-408a-958b-450bf8eed2fd"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:18:21.909154 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.909121    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9887975e-9a70-408a-958b-450bf8eed2fd-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9887975e-9a70-408a-958b-450bf8eed2fd" (UID: "9887975e-9a70-408a-958b-450bf8eed2fd"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:18:21.910645 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.910619    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9887975e-9a70-408a-958b-450bf8eed2fd-kube-api-access-4b6dv" (OuterVolumeSpecName: "kube-api-access-4b6dv") pod "9887975e-9a70-408a-958b-450bf8eed2fd" (UID: "9887975e-9a70-408a-958b-450bf8eed2fd"). InnerVolumeSpecName "kube-api-access-4b6dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:18:21.910750 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:21.910661    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9887975e-9a70-408a-958b-450bf8eed2fd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9887975e-9a70-408a-958b-450bf8eed2fd" (UID: "9887975e-9a70-408a-958b-450bf8eed2fd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:18:22.010272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.010187    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-cabundle-cert\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:22.010272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.010218    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9887975e-9a70-408a-958b-450bf8eed2fd-kserve-provision-location\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:22.010272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.010228    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b6dv\" (UniqueName: \"kubernetes.io/projected/9887975e-9a70-408a-958b-450bf8eed2fd-kube-api-access-4b6dv\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:22.010272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.010238    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9887975e-9a70-408a-958b-450bf8eed2fd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:22.010272 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.010247    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9887975e-9a70-408a-958b-450bf8eed2fd-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:18:22.195768 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.195736    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341bbd2a-973f-442d-8458-c3c633781933" path="/var/lib/kubelet/pods/341bbd2a-973f-442d-8458-c3c633781933/volumes"
Jul 09 18:18:22.228290 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.228253    2567 generic.go:356] "Generic (PLEG): container finished" podID="9887975e-9a70-408a-958b-450bf8eed2fd" containerID="f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b" exitCode=0
Jul 09 18:18:22.228464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.228309    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"
Jul 09 18:18:22.228464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.228336    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerDied","Data":"f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b"}
Jul 09 18:18:22.228464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.228400    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr" event={"ID":"9887975e-9a70-408a-958b-450bf8eed2fd","Type":"ContainerDied","Data":"a1e400670506c1173ab7dd246e86dbf716d69c8452e00792474c53e55756b60c"}
Jul 09 18:18:22.228464 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.228428    2567 scope.go:117] "RemoveContainer" containerID="30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4"
Jul 09 18:18:22.237136 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.237119    2567 scope.go:117] "RemoveContainer" containerID="f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b"
Jul 09 18:18:22.245423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.245402    2567 scope.go:117] "RemoveContainer" containerID="0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01"
Jul 09 18:18:22.246425 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.246407    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"]
Jul 09 18:18:22.252268 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.252248    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7595977785-zw5wr"]
Jul 09 18:18:22.253720 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.253705    2567 scope.go:117] "RemoveContainer" containerID="30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4"
Jul 09 18:18:22.254033 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:22.254012    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4\": container with ID starting with 30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4 not found: ID does not exist" containerID="30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4"
Jul 09 18:18:22.254080 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.254042    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4"} err="failed to get container status \"30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4\": rpc error: code = NotFound desc = could not find container \"30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4\": container with ID starting with 30bf9d60d074fc473ac3e6a77a091daae660274e0c38069744bac1fc19c5f1c4 not found: ID does not exist"
Jul 09 18:18:22.254080 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.254062    2567 scope.go:117] "RemoveContainer" containerID="f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b"
Jul 09 18:18:22.254321 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:22.254303    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b\": container with ID starting with f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b not found: ID does not exist" containerID="f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b"
Jul 09 18:18:22.254379 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.254326    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b"} err="failed to get container status \"f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b\": rpc error: code = NotFound desc = could not find container \"f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b\": container with ID starting with f4bbd403ce39e3f9ddbd5f6050dd7cddca1c49050952f86853b559790b1a337b not found: ID does not exist"
Jul 09 18:18:22.254379 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.254342    2567 scope.go:117] "RemoveContainer" containerID="0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01"
Jul 09 18:18:22.254567 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:18:22.254550    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01\": container with ID starting with 0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01 not found: ID does not exist" containerID="0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01"
Jul 09 18:18:22.254610 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:22.254572    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01"} err="failed to get container status \"0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01\": rpc error: code = NotFound desc = could not find container \"0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01\": container with ID starting with 0f55a45ff79edca976f8dc02c2d95a4066bf884cabc60ac4834cbee0f828cd01 not found: ID does not exist"
Jul 09 18:18:24.195248 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:24.195218    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" path="/var/lib/kubelet/pods/9887975e-9a70-408a-958b-450bf8eed2fd/volumes"
Jul 09 18:18:24.213820 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:24.213775    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:18:24.214259 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:24.214232    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:18:29.034930 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:29.034899    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:18:34.214479 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:34.214437    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:18:40.927210 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:40.927167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:41.384137 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:41.384094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:43.016760 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:43.016722    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:43.072078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:43.072050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:44.215242 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:44.215207    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:18:54.215034 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:54.214949    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:18:58.800533 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:58.800430    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:18:58.810380 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:18:58.802868    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:19:04.214759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:19:04.214727    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:19:16.836580 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:19:16.836548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:18.271311 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:19:18.271276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:30.597727 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:19:30.597684    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:00.294556 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:20:00.294508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:27.988219 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:21:27.988177    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:35.661357 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:21:35.661318    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:58.832600 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:23:58.832484    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:23:58.837768 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:23:58.837748    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:26:43.300945 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:43.300847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:55.533917 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.533884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:55.661913 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.661858    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"]
Jul 09 18:26:55.662169 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.662137    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container" containerID="cri-o://b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52" gracePeriod=30
Jul 09 18:26:55.662423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.662168    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kube-rbac-proxy" containerID="cri-o://f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f" gracePeriod=30
Jul 09 18:26:55.691675 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.691652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:55.741954 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.741918    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"]
Jul 09 18:26:55.742541 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742527    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container"
Jul 09 18:26:55.742584 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742544    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container"
Jul 09 18:26:55.742584 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742561    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="storage-initializer"
Jul 09 18:26:55.742584 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742570    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="storage-initializer"
Jul 09 18:26:55.742584 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742580    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="storage-initializer"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742589    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="storage-initializer"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742603    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742612    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742638    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kube-rbac-proxy"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742646    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kube-rbac-proxy"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742657    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kube-rbac-proxy"
Jul 09 18:26:55.742708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742666    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kube-rbac-proxy"
Jul 09 18:26:55.742934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742759    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kube-rbac-proxy"
Jul 09 18:26:55.742934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742769    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kube-rbac-proxy"
Jul 09 18:26:55.742934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742797    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9887975e-9a70-408a-958b-450bf8eed2fd" containerName="kserve-container"
Jul 09 18:26:55.742934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.742808    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="341bbd2a-973f-442d-8458-c3c633781933" containerName="kserve-container"
Jul 09 18:26:55.746614 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.746587    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.750357 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.750330    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-64fa7-kube-rbac-proxy-sar-config\""
Jul 09 18:26:55.750636 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.750615    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-64fa7-predictor-serving-cert\""
Jul 09 18:26:55.763274 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.762898    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"]
Jul 09 18:26:55.776399 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.776372    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:55.804355 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.804290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7bm4\" (UniqueName: \"kubernetes.io/projected/642bb608-0635-43ca-af1c-a65418f7cb56-kube-api-access-c7bm4\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.804481 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.804384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.804481 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.804455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/642bb608-0635-43ca-af1c-a65418f7cb56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.867663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.867633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:55.905029 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.904995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7bm4\" (UniqueName: \"kubernetes.io/projected/642bb608-0635-43ca-af1c-a65418f7cb56-kube-api-access-c7bm4\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.905176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.905034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.905176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.905087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/642bb608-0635-43ca-af1c-a65418f7cb56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.905306 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:26:55.905228    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-serving-cert: secret "success-200-isvc-64fa7-predictor-serving-cert" not found
Jul 09 18:26:55.905306 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:26:55.905292    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls podName:642bb608-0635-43ca-af1c-a65418f7cb56 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:56.405272572 +0000 UTC m=+1378.752180479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls") pod "success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" (UID: "642bb608-0635-43ca-af1c-a65418f7cb56") : secret "success-200-isvc-64fa7-predictor-serving-cert" not found
Jul 09 18:26:55.905757 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.905732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/642bb608-0635-43ca-af1c-a65418f7cb56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:55.914745 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:55.914723    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7bm4\" (UniqueName: \"kubernetes.io/projected/642bb608-0635-43ca-af1c-a65418f7cb56-kube-api-access-c7bm4\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:56.010748 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.010705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:56.326320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.326281    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:56.409197 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.409158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:56.411609 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.411577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls\") pod \"success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") " pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:56.663454 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.663367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:56.716394 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.716358    2567 generic.go:356] "Generic (PLEG): container finished" podID="346318b7-30ec-4943-90dd-bf4f8583f156" containerID="f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f" exitCode=2
Jul 09 18:26:56.716570 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.716435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" event={"ID":"346318b7-30ec-4943-90dd-bf4f8583f156","Type":"ContainerDied","Data":"f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f"}
Jul 09 18:26:56.802847 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:56.802821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"]
Jul 09 18:26:56.805219 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:26:56.805193    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642bb608_0635_43ca_af1c_a65418f7cb56.slice/crio-27b4300f488e7cc13a969c08dbd64fd0e75aba1179f03aa92fa21a7ef774f24f WatchSource:0}: Error finding container 27b4300f488e7cc13a969c08dbd64fd0e75aba1179f03aa92fa21a7ef774f24f: Status 404 returned error can't find the container with id 27b4300f488e7cc13a969c08dbd64fd0e75aba1179f03aa92fa21a7ef774f24f
Jul 09 18:26:57.723823 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:57.723759    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" event={"ID":"642bb608-0635-43ca-af1c-a65418f7cb56","Type":"ContainerStarted","Data":"b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2"}
Jul 09 18:26:57.724314 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:57.723891    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:57.724314 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:57.723930    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" event={"ID":"642bb608-0635-43ca-af1c-a65418f7cb56","Type":"ContainerStarted","Data":"803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513"}
Jul 09 18:26:57.724314 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:57.723964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" event={"ID":"642bb608-0635-43ca-af1c-a65418f7cb56","Type":"ContainerStarted","Data":"27b4300f488e7cc13a969c08dbd64fd0e75aba1179f03aa92fa21a7ef774f24f"}
Jul 09 18:26:57.743568 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:57.743516    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podStartSLOduration=2.743500127 podStartE2EDuration="2.743500127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:57.742235846 +0000 UTC m=+1380.089143773" watchObservedRunningTime="2026-07-09 18:26:57.743500127 +0000 UTC m=+1380.090408043"
Jul 09 18:26:58.728759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:58.728729    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:26:58.729951 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:58.729925    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 09 18:26:58.919708 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:58.919678    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:26:59.030274 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.030234    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/346318b7-30ec-4943-90dd-bf4f8583f156-kube-rbac-proxy-sar-config\") pod \"346318b7-30ec-4943-90dd-bf4f8583f156\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") "
Jul 09 18:26:59.030460 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.030289    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb2dw\" (UniqueName: \"kubernetes.io/projected/346318b7-30ec-4943-90dd-bf4f8583f156-kube-api-access-nb2dw\") pod \"346318b7-30ec-4943-90dd-bf4f8583f156\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") "
Jul 09 18:26:59.030460 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.030335    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls\") pod \"346318b7-30ec-4943-90dd-bf4f8583f156\" (UID: \"346318b7-30ec-4943-90dd-bf4f8583f156\") "
Jul 09 18:26:59.030701 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.030675    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346318b7-30ec-4943-90dd-bf4f8583f156-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "346318b7-30ec-4943-90dd-bf4f8583f156" (UID: "346318b7-30ec-4943-90dd-bf4f8583f156"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:26:59.032487 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.032462    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "346318b7-30ec-4943-90dd-bf4f8583f156" (UID: "346318b7-30ec-4943-90dd-bf4f8583f156"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:26:59.032600 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.032485    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346318b7-30ec-4943-90dd-bf4f8583f156-kube-api-access-nb2dw" (OuterVolumeSpecName: "kube-api-access-nb2dw") pod "346318b7-30ec-4943-90dd-bf4f8583f156" (UID: "346318b7-30ec-4943-90dd-bf4f8583f156"). InnerVolumeSpecName "kube-api-access-nb2dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:26:59.131256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.131218    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb2dw\" (UniqueName: \"kubernetes.io/projected/346318b7-30ec-4943-90dd-bf4f8583f156-kube-api-access-nb2dw\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:26:59.131256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.131249    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/346318b7-30ec-4943-90dd-bf4f8583f156-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:26:59.131256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.131260    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/346318b7-30ec-4943-90dd-bf4f8583f156-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:26:59.734610 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.734575    2567 generic.go:356] "Generic (PLEG): container finished" podID="346318b7-30ec-4943-90dd-bf4f8583f156" containerID="b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52" exitCode=0
Jul 09 18:26:59.735047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.734631    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"
Jul 09 18:26:59.735047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.734661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" event={"ID":"346318b7-30ec-4943-90dd-bf4f8583f156","Type":"ContainerDied","Data":"b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52"}
Jul 09 18:26:59.735047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.734732    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn" event={"ID":"346318b7-30ec-4943-90dd-bf4f8583f156","Type":"ContainerDied","Data":"8e645d77978072b20895d575d54befd732847873a9f6552529883246fbf4dd6f"}
Jul 09 18:26:59.735047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.734769    2567 scope.go:117] "RemoveContainer" containerID="f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f"
Jul 09 18:26:59.735402 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.735381    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 09 18:26:59.744484 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.744465    2567 scope.go:117] "RemoveContainer" containerID="b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52"
Jul 09 18:26:59.752525 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.752505    2567 scope.go:117] "RemoveContainer" containerID="f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f"
Jul 09 18:26:59.752808 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:26:59.752770    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f\": container with ID starting with f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f not found: ID does not exist" containerID="f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f"
Jul 09 18:26:59.752864 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.752819    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f"} err="failed to get container status \"f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f\": rpc error: code = NotFound desc = could not find container \"f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f\": container with ID starting with f54af20f6d8b4870265aa3d9257b199f037be215e3018b523417a247150d644f not found: ID does not exist"
Jul 09 18:26:59.752864 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.752837    2567 scope.go:117] "RemoveContainer" containerID="b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52"
Jul 09 18:26:59.753090 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:26:59.753072    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52\": container with ID starting with b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52 not found: ID does not exist" containerID="b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52"
Jul 09 18:26:59.753130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.753097    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52"} err="failed to get container status \"b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52\": rpc error: code = NotFound desc = could not find container \"b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52\": container with ID starting with b9fe516cd4dbe5db2c450b4b5ffcd0ce51e5b0b948f3aaa889af6dcd66dabe52 not found: ID does not exist"
Jul 09 18:26:59.756465 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.756442    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"]
Jul 09 18:26:59.760466 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:26:59.760444    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2a7bf-predictor-8694b45ccf-tqlbn"]
Jul 09 18:27:00.194780 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:00.194747    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" path="/var/lib/kubelet/pods/346318b7-30ec-4943-90dd-bf4f8583f156/volumes"
Jul 09 18:27:04.740179 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:04.740142    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:27:04.740834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:04.740807    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 09 18:27:14.741617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:14.741572    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 09 18:27:24.741545 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:24.741499    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 09 18:27:31.589066 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.589016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:31.706406 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.706349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:31.808653 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.808608    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"]
Jul 09 18:27:31.808924 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.808875    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" containerID="cri-o://55c2a1d5302f64470c87296738a8970d28473e602d56337eaa91b75dcb68098f" gracePeriod=30
Jul 09 18:27:31.809028 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.808928    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kube-rbac-proxy" containerID="cri-o://d2ffc18030f7dbf029a327ec5c1028aadfd7cf6e6d05e11576ba9eef07fee3ee" gracePeriod=30
Jul 09 18:27:31.885824 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.885718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:31.929963 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.929931    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"]
Jul 09 18:27:31.930567 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.930552    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container"
Jul 09 18:27:31.930611 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.930570    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container"
Jul 09 18:27:31.930611 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.930588    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kube-rbac-proxy"
Jul 09 18:27:31.930611 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.930597    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kube-rbac-proxy"
Jul 09 18:27:31.930709 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.930673    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kserve-container"
Jul 09 18:27:31.930709 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.930691    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="346318b7-30ec-4943-90dd-bf4f8583f156" containerName="kube-rbac-proxy"
Jul 09 18:27:31.934211 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.934185    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:31.936961 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.936932    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4dc46-predictor-serving-cert\""
Jul 09 18:27:31.937226 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.937209    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4dc46-kube-rbac-proxy-sar-config\""
Jul 09 18:27:31.950622 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:31.950575    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"]
Jul 09 18:27:32.014734 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.014694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:32.023892 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.023851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.023892 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.023888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.024085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.023929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlhdg\" (UniqueName: \"kubernetes.io/projected/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-api-access-wlhdg\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.124595 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.124555    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.124595 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.124596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.124879 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.124622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlhdg\" (UniqueName: \"kubernetes.io/projected/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-api-access-wlhdg\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.124879 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:27:32.124734    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-serving-cert: secret "success-200-isvc-4dc46-predictor-serving-cert" not found
Jul 09 18:27:32.124879 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:27:32.124838    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls podName:d6363f3d-1164-4837-a0d0-fa9b5fbc3667 nodeName:}" failed. No retries permitted until 2026-07-09 18:27:32.624815659 +0000 UTC m=+1414.971723557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls") pod "success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" (UID: "d6363f3d-1164-4837-a0d0-fa9b5fbc3667") : secret "success-200-isvc-4dc46-predictor-serving-cert" not found
Jul 09 18:27:32.125439 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.125404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.133427 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.133403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlhdg\" (UniqueName: \"kubernetes.io/projected/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-api-access-wlhdg\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.379270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.379238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:32.471838 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.471798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:32.629369 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.629278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.631689 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.631664    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls\") pod \"success-200-isvc-4dc46-predictor-6f7765d5-mwzcf\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") " pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.853451 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.853412    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:32.916584 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.916548    2567 generic.go:356] "Generic (PLEG): container finished" podID="7125cef7-4588-486c-9464-695fffa08ba8" containerID="d2ffc18030f7dbf029a327ec5c1028aadfd7cf6e6d05e11576ba9eef07fee3ee" exitCode=2
Jul 09 18:27:32.916742 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.916617    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" event={"ID":"7125cef7-4588-486c-9464-695fffa08ba8","Type":"ContainerDied","Data":"d2ffc18030f7dbf029a327ec5c1028aadfd7cf6e6d05e11576ba9eef07fee3ee"}
Jul 09 18:27:32.998393 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:32.998367    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"]
Jul 09 18:27:33.001066 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:27:33.001039    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6363f3d_1164_4837_a0d0_fa9b5fbc3667.slice/crio-eacf772c27ed7485b068c326f2188accc964332ca69d0ab1622b344357fa3d91 WatchSource:0}: Error finding container eacf772c27ed7485b068c326f2188accc964332ca69d0ab1622b344357fa3d91: Status 404 returned error can't find the container with id eacf772c27ed7485b068c326f2188accc964332ca69d0ab1622b344357fa3d91
Jul 09 18:27:33.923260 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.923222    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" event={"ID":"d6363f3d-1164-4837-a0d0-fa9b5fbc3667","Type":"ContainerStarted","Data":"ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0"}
Jul 09 18:27:33.923686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.923283    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" event={"ID":"d6363f3d-1164-4837-a0d0-fa9b5fbc3667","Type":"ContainerStarted","Data":"27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e"}
Jul 09 18:27:33.923686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.923327    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:33.923686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.923370    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:33.923686 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.923390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" event={"ID":"d6363f3d-1164-4837-a0d0-fa9b5fbc3667","Type":"ContainerStarted","Data":"eacf772c27ed7485b068c326f2188accc964332ca69d0ab1622b344357fa3d91"}
Jul 09 18:27:33.924954 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.924931    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 09 18:27:33.941075 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:33.941029    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podStartSLOduration=2.941012847 podStartE2EDuration="2.941012847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:33.939877109 +0000 UTC m=+1416.286785025" watchObservedRunningTime="2026-07-09 18:27:33.941012847 +0000 UTC m=+1416.287920768"
Jul 09 18:27:34.210013 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:34.209973    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.40:8643/healthz\": dial tcp 10.132.0.40:8643: connect: connection refused"
Jul 09 18:27:34.214292 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:34.214257    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 09 18:27:34.741494 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:34.741448    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 09 18:27:34.930593 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:34.930563    2567 generic.go:356] "Generic (PLEG): container finished" podID="7125cef7-4588-486c-9464-695fffa08ba8" containerID="55c2a1d5302f64470c87296738a8970d28473e602d56337eaa91b75dcb68098f" exitCode=0
Jul 09 18:27:34.930985 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:34.930644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" event={"ID":"7125cef7-4588-486c-9464-695fffa08ba8","Type":"ContainerDied","Data":"55c2a1d5302f64470c87296738a8970d28473e602d56337eaa91b75dcb68098f"}
Jul 09 18:27:34.931056 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:34.931027    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 09 18:27:35.063506 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.063482    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:27:35.151441 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.151412    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7125cef7-4588-486c-9464-695fffa08ba8-kube-rbac-proxy-sar-config\") pod \"7125cef7-4588-486c-9464-695fffa08ba8\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") "
Jul 09 18:27:35.151581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.151454    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z55xp\" (UniqueName: \"kubernetes.io/projected/7125cef7-4588-486c-9464-695fffa08ba8-kube-api-access-z55xp\") pod \"7125cef7-4588-486c-9464-695fffa08ba8\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") "
Jul 09 18:27:35.151581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.151482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls\") pod \"7125cef7-4588-486c-9464-695fffa08ba8\" (UID: \"7125cef7-4588-486c-9464-695fffa08ba8\") "
Jul 09 18:27:35.151747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.151724    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7125cef7-4588-486c-9464-695fffa08ba8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7125cef7-4588-486c-9464-695fffa08ba8" (UID: "7125cef7-4588-486c-9464-695fffa08ba8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:35.153589 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.153559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7125cef7-4588-486c-9464-695fffa08ba8" (UID: "7125cef7-4588-486c-9464-695fffa08ba8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:35.153706 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.153590    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7125cef7-4588-486c-9464-695fffa08ba8-kube-api-access-z55xp" (OuterVolumeSpecName: "kube-api-access-z55xp") pod "7125cef7-4588-486c-9464-695fffa08ba8" (UID: "7125cef7-4588-486c-9464-695fffa08ba8"). InnerVolumeSpecName "kube-api-access-z55xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:27:35.252566 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.252522    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7125cef7-4588-486c-9464-695fffa08ba8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:27:35.252566 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.252556    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z55xp\" (UniqueName: \"kubernetes.io/projected/7125cef7-4588-486c-9464-695fffa08ba8-kube-api-access-z55xp\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:27:35.252566 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.252570    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7125cef7-4588-486c-9464-695fffa08ba8-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:27:35.937590 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.937554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4" event={"ID":"7125cef7-4588-486c-9464-695fffa08ba8","Type":"ContainerDied","Data":"25e92fb1fad1f77aa8e4672e10e6d525de352a255a39460247e3fdd8a2e756ce"}
Jul 09 18:27:35.938016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.937622    2567 scope.go:117] "RemoveContainer" containerID="d2ffc18030f7dbf029a327ec5c1028aadfd7cf6e6d05e11576ba9eef07fee3ee"
Jul 09 18:27:35.938016 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.937565    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"
Jul 09 18:27:35.947047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.947032    2567 scope.go:117] "RemoveContainer" containerID="55c2a1d5302f64470c87296738a8970d28473e602d56337eaa91b75dcb68098f"
Jul 09 18:27:35.961011 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.960989    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"]
Jul 09 18:27:35.965203 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:35.965180    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e8119-predictor-7cf49bdfbc-v9kj4"]
Jul 09 18:27:36.194990 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:36.194960    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7125cef7-4588-486c-9464-695fffa08ba8" path="/var/lib/kubelet/pods/7125cef7-4588-486c-9464-695fffa08ba8/volumes"
Jul 09 18:27:39.935428 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:39.935399    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:27:39.935864 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:39.935815    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 09 18:27:44.742384 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:44.742346    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:27:49.936035 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:49.935992    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 09 18:27:56.678287 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:56.678200    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:56.746589 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:56.746555    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:59.936663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:27:59.936621    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 09 18:28:05.925279 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:05.925236    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:06.016402 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.016359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:06.044565 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.044533    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"]
Jul 09 18:28:06.044944 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.044907    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container" containerID="cri-o://803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513" gracePeriod=30
Jul 09 18:28:06.045073 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.045014    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kube-rbac-proxy" containerID="cri-o://b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2" gracePeriod=30
Jul 09 18:28:06.101906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.101863    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:06.110479 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110450    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"]
Jul 09 18:28:06.110888 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110875    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kube-rbac-proxy"
Jul 09 18:28:06.110888 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110889    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kube-rbac-proxy"
Jul 09 18:28:06.110980 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110900    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container"
Jul 09 18:28:06.110980 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110906    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container"
Jul 09 18:28:06.110980 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110963    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kube-rbac-proxy"
Jul 09 18:28:06.110980 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.110975    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7125cef7-4588-486c-9464-695fffa08ba8" containerName="kserve-container"
Jul 09 18:28:06.114446 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.114429    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.117244 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.117222    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f76a1-kube-rbac-proxy-sar-config\""
Jul 09 18:28:06.117367 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.117230    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f76a1-predictor-serving-cert\""
Jul 09 18:28:06.132813 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.132760    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"]
Jul 09 18:28:06.227250 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.227212    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:06.234233 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.234207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-proxy-tls\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.234355 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.234269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.234355 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.234339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h8kq\" (UniqueName: \"kubernetes.io/projected/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-api-access-2h8kq\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.322450 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.322416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:06.334945 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.334912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.335115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.334953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h8kq\" (UniqueName: \"kubernetes.io/projected/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-api-access-2h8kq\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.335115 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.335025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-proxy-tls\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.335764 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.335742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.337467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.337448    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-proxy-tls\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.344765 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.344739    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h8kq\" (UniqueName: \"kubernetes.io/projected/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-api-access-2h8kq\") pod \"success-200-isvc-f76a1-predictor-79d744db9c-fsz5s\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") " pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.424706 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.424665    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:06.600184 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.600086    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"]
Jul 09 18:28:06.603064 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:28:06.603034    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b781ef_c13f_4c4d_9b0b_99d97012f6fe.slice/crio-040aa65eeb77555c2a7873b428dc172c811b3c3aec2901a3bd778279cf85e532 WatchSource:0}: Error finding container 040aa65eeb77555c2a7873b428dc172c811b3c3aec2901a3bd778279cf85e532: Status 404 returned error can't find the container with id 040aa65eeb77555c2a7873b428dc172c811b3c3aec2901a3bd778279cf85e532
Jul 09 18:28:06.647924 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:06.647895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:07.096699 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.096590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" event={"ID":"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe","Type":"ContainerStarted","Data":"5b29787b461732d67b0fdb33511fee6cab00c1c82da97be69245114234506450"}
Jul 09 18:28:07.096699 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.096656    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:07.096699 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.096687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" event={"ID":"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe","Type":"ContainerStarted","Data":"5ab1442fc0d80b28c74d773df7336fbff4cb0d84d23e31e08a3b9ac18123112e"}
Jul 09 18:28:07.097244 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.096720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" event={"ID":"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe","Type":"ContainerStarted","Data":"040aa65eeb77555c2a7873b428dc172c811b3c3aec2901a3bd778279cf85e532"}
Jul 09 18:28:07.098458 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.098435    2567 generic.go:356] "Generic (PLEG): container finished" podID="642bb608-0635-43ca-af1c-a65418f7cb56" containerID="b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2" exitCode=2
Jul 09 18:28:07.098545 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.098478    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" event={"ID":"642bb608-0635-43ca-af1c-a65418f7cb56","Type":"ContainerDied","Data":"b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2"}
Jul 09 18:28:07.124204 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:07.124160    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podStartSLOduration=1.124143068 podStartE2EDuration="1.124143068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:07.121511316 +0000 UTC m=+1449.468419251" watchObservedRunningTime="2026-07-09 18:28:07.124143068 +0000 UTC m=+1449.471051054"
Jul 09 18:28:08.104029 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:08.103998    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:08.105107 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:08.105081    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 09 18:28:09.108777 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.108745    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 09 18:28:09.314693 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.314671    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:28:09.461127 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.461096    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/642bb608-0635-43ca-af1c-a65418f7cb56-kube-rbac-proxy-sar-config\") pod \"642bb608-0635-43ca-af1c-a65418f7cb56\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") "
Jul 09 18:28:09.461127 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.461131    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7bm4\" (UniqueName: \"kubernetes.io/projected/642bb608-0635-43ca-af1c-a65418f7cb56-kube-api-access-c7bm4\") pod \"642bb608-0635-43ca-af1c-a65418f7cb56\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") "
Jul 09 18:28:09.461331 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.461200    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls\") pod \"642bb608-0635-43ca-af1c-a65418f7cb56\" (UID: \"642bb608-0635-43ca-af1c-a65418f7cb56\") "
Jul 09 18:28:09.461537 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.461517    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/642bb608-0635-43ca-af1c-a65418f7cb56-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "642bb608-0635-43ca-af1c-a65418f7cb56" (UID: "642bb608-0635-43ca-af1c-a65418f7cb56"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:09.463363 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.463335    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "642bb608-0635-43ca-af1c-a65418f7cb56" (UID: "642bb608-0635-43ca-af1c-a65418f7cb56"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:09.463487 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.463412    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642bb608-0635-43ca-af1c-a65418f7cb56-kube-api-access-c7bm4" (OuterVolumeSpecName: "kube-api-access-c7bm4") pod "642bb608-0635-43ca-af1c-a65418f7cb56" (UID: "642bb608-0635-43ca-af1c-a65418f7cb56"). InnerVolumeSpecName "kube-api-access-c7bm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:09.562411 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.562367    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/642bb608-0635-43ca-af1c-a65418f7cb56-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:28:09.562411 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.562402    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7bm4\" (UniqueName: \"kubernetes.io/projected/642bb608-0635-43ca-af1c-a65418f7cb56-kube-api-access-c7bm4\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:28:09.562411 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.562416    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/642bb608-0635-43ca-af1c-a65418f7cb56-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:28:09.936433 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:09.936348    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 09 18:28:10.116087 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.116049    2567 generic.go:356] "Generic (PLEG): container finished" podID="642bb608-0635-43ca-af1c-a65418f7cb56" containerID="803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513" exitCode=0
Jul 09 18:28:10.116475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.116104    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"
Jul 09 18:28:10.116475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.116134    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" event={"ID":"642bb608-0635-43ca-af1c-a65418f7cb56","Type":"ContainerDied","Data":"803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513"}
Jul 09 18:28:10.116475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.116211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs" event={"ID":"642bb608-0635-43ca-af1c-a65418f7cb56","Type":"ContainerDied","Data":"27b4300f488e7cc13a969c08dbd64fd0e75aba1179f03aa92fa21a7ef774f24f"}
Jul 09 18:28:10.116475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.116249    2567 scope.go:117] "RemoveContainer" containerID="b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2"
Jul 09 18:28:10.131887 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.126024    2567 scope.go:117] "RemoveContainer" containerID="803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513"
Jul 09 18:28:10.140352 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.140330    2567 scope.go:117] "RemoveContainer" containerID="b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2"
Jul 09 18:28:10.140632 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:28:10.140611    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2\": container with ID starting with b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2 not found: ID does not exist" containerID="b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2"
Jul 09 18:28:10.140679 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.140641    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2"} err="failed to get container status \"b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2\": rpc error: code = NotFound desc = could not find container \"b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2\": container with ID starting with b49e6f7a163ac6e7ef84cde381c64b71f882c66ca3abc2df08cf626f7bb54cd2 not found: ID does not exist"
Jul 09 18:28:10.140679 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.140660    2567 scope.go:117] "RemoveContainer" containerID="803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513"
Jul 09 18:28:10.140989 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:28:10.140975    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513\": container with ID starting with 803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513 not found: ID does not exist" containerID="803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513"
Jul 09 18:28:10.141049 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.140991    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513"} err="failed to get container status \"803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513\": rpc error: code = NotFound desc = could not find container \"803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513\": container with ID starting with 803d322497965cadbb886af71a9cc8592f00fddb6f295540c3902c3d2d7d2513 not found: ID does not exist"
Jul 09 18:28:10.145499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.145477    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"]
Jul 09 18:28:10.155748 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.155723    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-64fa7-predictor-64bc9d87f9-c22hs"]
Jul 09 18:28:10.197685 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:10.197653    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" path="/var/lib/kubelet/pods/642bb608-0635-43ca-af1c-a65418f7cb56/volumes"
Jul 09 18:28:14.113429 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:14.113400    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:14.113900 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:14.113871    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 09 18:28:19.936776 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:19.936748    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:28:24.114328 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:24.114286    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 09 18:28:31.839934 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:31.839901    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:31.994850 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:31.994808    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:34.114592 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:34.114554    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 09 18:28:41.976901 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:41.976864    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:42.101078 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.101038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:42.180499 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.180471    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"]
Jul 09 18:28:42.180745 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.180720    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container" containerID="cri-o://27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e" gracePeriod=30
Jul 09 18:28:42.180880 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.180819    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kube-rbac-proxy" containerID="cri-o://ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0" gracePeriod=30
Jul 09 18:28:42.228905 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.228851    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"]
Jul 09 18:28:42.229241 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.229229    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container"
Jul 09 18:28:42.229284 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.229243    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container"
Jul 09 18:28:42.229284 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.229255    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kube-rbac-proxy"
Jul 09 18:28:42.229284 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.229261    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kube-rbac-proxy"
Jul 09 18:28:42.229375 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.229334    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kserve-container"
Jul 09 18:28:42.229375 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.229343    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="642bb608-0635-43ca-af1c-a65418f7cb56" containerName="kube-rbac-proxy"
Jul 09 18:28:42.232752 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.232736    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.236586 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.236564    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2ff98-predictor-serving-cert\""
Jul 09 18:28:42.236586 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.236575    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2ff98-kube-rbac-proxy-sar-config\""
Jul 09 18:28:42.239018 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.238993    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:42.249679 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.249657    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"]
Jul 09 18:28:42.331476 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.331441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-proxy-tls\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.331652 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.331526    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.331652 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.331584    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td4jv\" (UniqueName: \"kubernetes.io/projected/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-api-access-td4jv\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.383906 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.383857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:42.432565 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.432529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-proxy-tls\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.432748 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.432598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.432748 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.432664    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td4jv\" (UniqueName: \"kubernetes.io/projected/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-api-access-td4jv\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.433403 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.433371    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.435100 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.435073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-proxy-tls\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.455747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.455724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td4jv\" (UniqueName: \"kubernetes.io/projected/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-api-access-td4jv\") pod \"success-200-isvc-2ff98-predictor-85dc97599c-ldrd2\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") " pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.486106 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.486024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:42.543316 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.543285    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:42.680195 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.680150    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"]
Jul 09 18:28:42.683844 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:28:42.683811    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6acb0f6_b6db_4974_84f9_ad9e6f6a3001.slice/crio-72a9cd479fae52f238ff7ae2259db992bad1369a0a5ea275bc1439e562dae289 WatchSource:0}: Error finding container 72a9cd479fae52f238ff7ae2259db992bad1369a0a5ea275bc1439e562dae289: Status 404 returned error can't find the container with id 72a9cd479fae52f238ff7ae2259db992bad1369a0a5ea275bc1439e562dae289
Jul 09 18:28:42.818938 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:42.818908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:43.284608 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.284568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" event={"ID":"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001","Type":"ContainerStarted","Data":"1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7"}
Jul 09 18:28:43.285114 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.284645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" event={"ID":"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001","Type":"ContainerStarted","Data":"e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0"}
Jul 09 18:28:43.285114 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.284680    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:43.285114 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.284707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" event={"ID":"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001","Type":"ContainerStarted","Data":"72a9cd479fae52f238ff7ae2259db992bad1369a0a5ea275bc1439e562dae289"}
Jul 09 18:28:43.286825 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.286797    2567 generic.go:356] "Generic (PLEG): container finished" podID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerID="ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0" exitCode=2
Jul 09 18:28:43.286932 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.286816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" event={"ID":"d6363f3d-1164-4837-a0d0-fa9b5fbc3667","Type":"ContainerDied","Data":"ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0"}
Jul 09 18:28:43.310769 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:43.310721    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podStartSLOduration=1.310703978 podStartE2EDuration="1.310703978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:43.30819492 +0000 UTC m=+1485.655102839" watchObservedRunningTime="2026-07-09 18:28:43.310703978 +0000 UTC m=+1485.657611901"
Jul 09 18:28:44.114002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:44.113961    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 09 18:28:44.293656 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:44.293625    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:44.295074 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:44.295035    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:28:44.931257 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:44.931217    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.42:8643/healthz\": dial tcp 10.132.0.42:8643: connect: connection refused"
Jul 09 18:28:45.299014 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.298981    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:28:45.536360 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.536300    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:28:45.659999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.659965    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls\") pod \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") "
Jul 09 18:28:45.660164 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.660038    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlhdg\" (UniqueName: \"kubernetes.io/projected/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-api-access-wlhdg\") pod \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") "
Jul 09 18:28:45.660164 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.660075    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-rbac-proxy-sar-config\") pod \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\" (UID: \"d6363f3d-1164-4837-a0d0-fa9b5fbc3667\") "
Jul 09 18:28:45.660438 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.660415    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d6363f3d-1164-4837-a0d0-fa9b5fbc3667" (UID: "d6363f3d-1164-4837-a0d0-fa9b5fbc3667"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:45.662118 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.662094    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d6363f3d-1164-4837-a0d0-fa9b5fbc3667" (UID: "d6363f3d-1164-4837-a0d0-fa9b5fbc3667"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:45.662196 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.662132    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-api-access-wlhdg" (OuterVolumeSpecName: "kube-api-access-wlhdg") pod "d6363f3d-1164-4837-a0d0-fa9b5fbc3667" (UID: "d6363f3d-1164-4837-a0d0-fa9b5fbc3667"). InnerVolumeSpecName "kube-api-access-wlhdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:45.760834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.760767    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlhdg\" (UniqueName: \"kubernetes.io/projected/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-api-access-wlhdg\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:28:45.760834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.760831    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:28:45.760834 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:45.760842    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6363f3d-1164-4837-a0d0-fa9b5fbc3667-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:28:46.305384 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.305346    2567 generic.go:356] "Generic (PLEG): container finished" podID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerID="27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e" exitCode=0
Jul 09 18:28:46.305771 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.305395    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" event={"ID":"d6363f3d-1164-4837-a0d0-fa9b5fbc3667","Type":"ContainerDied","Data":"27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e"}
Jul 09 18:28:46.305771 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.305455    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf" event={"ID":"d6363f3d-1164-4837-a0d0-fa9b5fbc3667","Type":"ContainerDied","Data":"eacf772c27ed7485b068c326f2188accc964332ca69d0ab1622b344357fa3d91"}
Jul 09 18:28:46.305771 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.305481    2567 scope.go:117] "RemoveContainer" containerID="ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0"
Jul 09 18:28:46.305771 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.305404    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"
Jul 09 18:28:46.314061 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.314046    2567 scope.go:117] "RemoveContainer" containerID="27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e"
Jul 09 18:28:46.321670 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.321655    2567 scope.go:117] "RemoveContainer" containerID="ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0"
Jul 09 18:28:46.321911 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:28:46.321892    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0\": container with ID starting with ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0 not found: ID does not exist" containerID="ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0"
Jul 09 18:28:46.321978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.321917    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0"} err="failed to get container status \"ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0\": rpc error: code = NotFound desc = could not find container \"ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0\": container with ID starting with ab4d4abcc7a24c9d1f35bd40c18d7c1000bb60e28e15b34fae91864e00a027a0 not found: ID does not exist"
Jul 09 18:28:46.321978 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.321933    2567 scope.go:117] "RemoveContainer" containerID="27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e"
Jul 09 18:28:46.322136 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:28:46.322114    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e\": container with ID starting with 27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e not found: ID does not exist" containerID="27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e"
Jul 09 18:28:46.322182 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.322144    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e"} err="failed to get container status \"27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e\": rpc error: code = NotFound desc = could not find container \"27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e\": container with ID starting with 27ce59cbb44f589424c8b53711248147a0e970cdcdb7a534c12c47dbc82e0c5e not found: ID does not exist"
Jul 09 18:28:46.328430 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.328408    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"]
Jul 09 18:28:46.331764 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:46.331745    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4dc46-predictor-6f7765d5-mwzcf"]
Jul 09 18:28:48.195620 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:48.195589    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" path="/var/lib/kubelet/pods/d6363f3d-1164-4837-a0d0-fa9b5fbc3667/volumes"
Jul 09 18:28:50.303222 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:50.303193    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:28:50.303691 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:50.303635    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:28:54.115019 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:54.114983    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:28:58.867204 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:58.867084    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:28:58.876857 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:28:58.876835    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:29:00.304656 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:00.304610    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:29:06.184304 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:06.184260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:06.322035 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:06.322001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:10.304170 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:10.304129    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:29:20.303780 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:20.303737    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:29:30.304610 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:30.304570    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:29:42.256724 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:42.256689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:42.356572 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:42.356539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:52.051300 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:29:52.051261    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:58.899344 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:33:58.899229    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:33:58.919674 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:33:58.919652    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:36:34.203683 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:36:34.203630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:20.675629 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.675581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:20.784514 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.784480    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"]
Jul 09 18:37:20.784763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.784709    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container" containerID="cri-o://5ab1442fc0d80b28c74d773df7336fbff4cb0d84d23e31e08a3b9ac18123112e" gracePeriod=30
Jul 09 18:37:20.784968 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.784913    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kube-rbac-proxy" containerID="cri-o://5b29787b461732d67b0fdb33511fee6cab00c1c82da97be69245114234506450" gracePeriod=30
Jul 09 18:37:20.818916 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.818882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:20.878404 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878368    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"]
Jul 09 18:37:20.878844 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878831    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kube-rbac-proxy"
Jul 09 18:37:20.878897 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878848    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kube-rbac-proxy"
Jul 09 18:37:20.878897 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878869    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container"
Jul 09 18:37:20.878897 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878876    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container"
Jul 09 18:37:20.879009 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878950    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kserve-container"
Jul 09 18:37:20.879009 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.878960    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6363f3d-1164-4837-a0d0-fa9b5fbc3667" containerName="kube-rbac-proxy"
Jul 09 18:37:20.882179 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.882155    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:20.884486 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.884457    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-eb3ff-predictor-serving-cert\""
Jul 09 18:37:20.884486 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.884457    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-eb3ff-kube-rbac-proxy-sar-config\""
Jul 09 18:37:20.892577 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.892551    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"]
Jul 09 18:37:20.919426 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.919398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:20.946885 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.946856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f83dc391-abbb-4240-bea5-dc1c707f5925-proxy-tls\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:20.947010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.946941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:20.947010 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:20.946992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k4wv\" (UniqueName: \"kubernetes.io/projected/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-api-access-6k4wv\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.015298 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.015246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:21.048142 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.048108    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f83dc391-abbb-4240-bea5-dc1c707f5925-proxy-tls\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.048319 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.048208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.048319 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.048273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k4wv\" (UniqueName: \"kubernetes.io/projected/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-api-access-6k4wv\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.048995 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.048975    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.050687 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.050668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f83dc391-abbb-4240-bea5-dc1c707f5925-proxy-tls\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.056518 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.056494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k4wv\" (UniqueName: \"kubernetes.io/projected/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-api-access-6k4wv\") pod \"success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") " pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.114493 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.114459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:21.197161 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.197072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.293987 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.293953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:21.338362 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.338339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"]
Jul 09 18:37:21.343515 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:37:21.343453    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83dc391_abbb_4240_bea5_dc1c707f5925.slice/crio-db20dda74d36ae778bfa3d16543202c6fe80e57fd8128decdda31e0953ce2a2d WatchSource:0}: Error finding container db20dda74d36ae778bfa3d16543202c6fe80e57fd8128decdda31e0953ce2a2d: Status 404 returned error can't find the container with id db20dda74d36ae778bfa3d16543202c6fe80e57fd8128decdda31e0953ce2a2d
Jul 09 18:37:21.819763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.819683    2567 generic.go:356] "Generic (PLEG): container finished" podID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerID="5b29787b461732d67b0fdb33511fee6cab00c1c82da97be69245114234506450" exitCode=2
Jul 09 18:37:21.819763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.819741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" event={"ID":"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe","Type":"ContainerDied","Data":"5b29787b461732d67b0fdb33511fee6cab00c1c82da97be69245114234506450"}
Jul 09 18:37:21.821301 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.821279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" event={"ID":"f83dc391-abbb-4240-bea5-dc1c707f5925","Type":"ContainerStarted","Data":"afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5"}
Jul 09 18:37:21.821411 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.821328    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" event={"ID":"f83dc391-abbb-4240-bea5-dc1c707f5925","Type":"ContainerStarted","Data":"023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45"}
Jul 09 18:37:21.821411 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.821358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" event={"ID":"f83dc391-abbb-4240-bea5-dc1c707f5925","Type":"ContainerStarted","Data":"db20dda74d36ae778bfa3d16543202c6fe80e57fd8128decdda31e0953ce2a2d"}
Jul 09 18:37:21.821484 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.821437    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:21.841728 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:21.841689    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podStartSLOduration=1.841676395 podStartE2EDuration="1.841676395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:37:21.840388471 +0000 UTC m=+2004.187296388" watchObservedRunningTime="2026-07-09 18:37:21.841676395 +0000 UTC m=+2004.188584311"
Jul 09 18:37:22.831042 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:22.831009    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:22.833027 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:22.832982    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 09 18:37:23.173670 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.173584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:23.836211 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.836173    2567 generic.go:356] "Generic (PLEG): container finished" podID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerID="5ab1442fc0d80b28c74d773df7336fbff4cb0d84d23e31e08a3b9ac18123112e" exitCode=0
Jul 09 18:37:23.836681 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.836352    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" event={"ID":"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe","Type":"ContainerDied","Data":"5ab1442fc0d80b28c74d773df7336fbff4cb0d84d23e31e08a3b9ac18123112e"}
Jul 09 18:37:23.836831 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.836803    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 09 18:37:23.946002 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.945978    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:37:23.975320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.975295    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-proxy-tls\") pod \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") "
Jul 09 18:37:23.975462 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.975334    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h8kq\" (UniqueName: \"kubernetes.io/projected/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-api-access-2h8kq\") pod \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") "
Jul 09 18:37:23.975462 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.975392    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-rbac-proxy-sar-config\") pod \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\" (UID: \"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe\") "
Jul 09 18:37:23.975807 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.975769    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" (UID: "e8b781ef-c13f-4c4d-9b0b-99d97012f6fe"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:37:23.977433 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.977399    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" (UID: "e8b781ef-c13f-4c4d-9b0b-99d97012f6fe"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:37:23.977548 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:23.977506    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-api-access-2h8kq" (OuterVolumeSpecName: "kube-api-access-2h8kq") pod "e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" (UID: "e8b781ef-c13f-4c4d-9b0b-99d97012f6fe"). InnerVolumeSpecName "kube-api-access-2h8kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:37:24.076150 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.076061    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:37:24.076150 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.076097    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:37:24.076150 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.076116    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h8kq\" (UniqueName: \"kubernetes.io/projected/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe-kube-api-access-2h8kq\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:37:24.842287 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.842260    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"
Jul 09 18:37:24.842287 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.842282    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s" event={"ID":"e8b781ef-c13f-4c4d-9b0b-99d97012f6fe","Type":"ContainerDied","Data":"040aa65eeb77555c2a7873b428dc172c811b3c3aec2901a3bd778279cf85e532"}
Jul 09 18:37:24.843263 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.842343    2567 scope.go:117] "RemoveContainer" containerID="5b29787b461732d67b0fdb33511fee6cab00c1c82da97be69245114234506450"
Jul 09 18:37:24.852011 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.851992    2567 scope.go:117] "RemoveContainer" containerID="5ab1442fc0d80b28c74d773df7336fbff4cb0d84d23e31e08a3b9ac18123112e"
Jul 09 18:37:24.859579 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.859557    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"]
Jul 09 18:37:24.863158 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:24.863136    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f76a1-predictor-79d744db9c-fsz5s"]
Jul 09 18:37:26.194827 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:26.194774    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" path="/var/lib/kubelet/pods/e8b781ef-c13f-4c4d-9b0b-99d97012f6fe/volumes"
Jul 09 18:37:28.841858 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:28.841822    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:37:28.842320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:28.842294    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 09 18:37:38.842398 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:38.842357    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 09 18:37:48.843294 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:48.843252    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 09 18:37:56.926705 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:56.926669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:57.039938 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.039906    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"]
Jul 09 18:37:57.040176 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.040148    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" containerID="cri-o://e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0" gracePeriod=30
Jul 09 18:37:57.040256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.040190    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kube-rbac-proxy" containerID="cri-o://1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7" gracePeriod=30
Jul 09 18:37:57.060993 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.060965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:57.125661 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.125621    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"]
Jul 09 18:37:57.126102 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.126085    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kube-rbac-proxy"
Jul 09 18:37:57.126200 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.126104    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kube-rbac-proxy"
Jul 09 18:37:57.126200 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.126125    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container"
Jul 09 18:37:57.126200 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.126133    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container"
Jul 09 18:37:57.126412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.126261    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kube-rbac-proxy"
Jul 09 18:37:57.126412 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.126274    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b781ef-c13f-4c4d-9b0b-99d97012f6fe" containerName="kserve-container"
Jul 09 18:37:57.129641 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.129618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.132050 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.132024    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-bed12-predictor-serving-cert\""
Jul 09 18:37:57.132158 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.132032    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-bed12-kube-rbac-proxy-sar-config\""
Jul 09 18:37:57.142570 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.142545    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"]
Jul 09 18:37:57.150627 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.150602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:57.229382 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.229349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:57.269691 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.269659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.269888 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.269712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmljj\" (UniqueName: \"kubernetes.io/projected/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-api-access-pmljj\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.269888 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.269874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.370552 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.370508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmljj\" (UniqueName: \"kubernetes.io/projected/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-api-access-pmljj\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.370759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.370617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.370759 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.370660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.370929 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:37:57.370769    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-bed12-predictor-serving-cert: secret "success-200-isvc-bed12-predictor-serving-cert" not found
Jul 09 18:37:57.370929 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:37:57.370845    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls podName:390d1d1c-ae2a-4732-95a8-107dda4f14df nodeName:}" failed. No retries permitted until 2026-07-09 18:37:57.870829708 +0000 UTC m=+2040.217737602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls") pod "success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" (UID: "390d1d1c-ae2a-4732-95a8-107dda4f14df") : secret "success-200-isvc-bed12-predictor-serving-cert" not found
Jul 09 18:37:57.371374 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.371350    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.379143 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.379113    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmljj\" (UniqueName: \"kubernetes.io/projected/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-api-access-pmljj\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.510588 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.510506    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:57.611665 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.611631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:57.875354 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.875262    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:57.877621 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:57.877602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls\") pod \"success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") " pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:58.009174 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:58.009143    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerID="1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7" exitCode=2
Jul 09 18:37:58.009553 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:58.009212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" event={"ID":"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001","Type":"ContainerDied","Data":"1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7"}
Jul 09 18:37:58.040390 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:58.040363    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:58.176382 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:58.176356    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"]
Jul 09 18:37:58.178475 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:37:58.178450    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390d1d1c_ae2a_4732_95a8_107dda4f14df.slice/crio-553ae34d5e4acdbedde0b6d9dec87679d4d73d7ea72fceb4b0be39681c205b5e WatchSource:0}: Error finding container 553ae34d5e4acdbedde0b6d9dec87679d4d73d7ea72fceb4b0be39681c205b5e: Status 404 returned error can't find the container with id 553ae34d5e4acdbedde0b6d9dec87679d4d73d7ea72fceb4b0be39681c205b5e
Jul 09 18:37:58.843238 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:58.843195    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 09 18:37:59.016218 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.016180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" event={"ID":"390d1d1c-ae2a-4732-95a8-107dda4f14df","Type":"ContainerStarted","Data":"75b1818bdb0d8e237711d7299c9d150193bdb2c33c2ad8e694b1a12ac2617e38"}
Jul 09 18:37:59.016663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.016273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" event={"ID":"390d1d1c-ae2a-4732-95a8-107dda4f14df","Type":"ContainerStarted","Data":"5f2eda0c06468906d56150fb84e13cf6df8c9e6c952f3bb98b839152cd4b88bd"}
Jul 09 18:37:59.016663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.016299    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:59.016663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.016321    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:37:59.016663 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.016339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" event={"ID":"390d1d1c-ae2a-4732-95a8-107dda4f14df","Type":"ContainerStarted","Data":"553ae34d5e4acdbedde0b6d9dec87679d4d73d7ea72fceb4b0be39681c205b5e"}
Jul 09 18:37:59.018041 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.018013    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 09 18:37:59.034694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:37:59.034618    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podStartSLOduration=2.034604203 podStartE2EDuration="2.034604203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:37:59.03295917 +0000 UTC m=+2041.379867088" watchObservedRunningTime="2026-07-09 18:37:59.034604203 +0000 UTC m=+2041.381512119"
Jul 09 18:38:00.022248 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.022201    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 09 18:38:00.299700 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.299608    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.44:8643/healthz\": dial tcp 10.132.0.44:8643: connect: connection refused"
Jul 09 18:38:00.303970 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.303937    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 09 18:38:00.404187 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.404162    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:38:00.599320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.599229    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-rbac-proxy-sar-config\") pod \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") "
Jul 09 18:38:00.599320 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.599280    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-proxy-tls\") pod \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") "
Jul 09 18:38:00.599533 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.599351    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td4jv\" (UniqueName: \"kubernetes.io/projected/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-api-access-td4jv\") pod \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\" (UID: \"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001\") "
Jul 09 18:38:00.599644 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.599622    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" (UID: "c6acb0f6-b6db-4974-84f9-ad9e6f6a3001"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:38:00.601325 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.601301    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" (UID: "c6acb0f6-b6db-4974-84f9-ad9e6f6a3001"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:38:00.601523 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.601506    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-api-access-td4jv" (OuterVolumeSpecName: "kube-api-access-td4jv") pod "c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" (UID: "c6acb0f6-b6db-4974-84f9-ad9e6f6a3001"). InnerVolumeSpecName "kube-api-access-td4jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:38:00.700986 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.700949    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td4jv\" (UniqueName: \"kubernetes.io/projected/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-api-access-td4jv\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:38:00.700986 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.700977    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:38:00.700986 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:00.700987    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:38:01.028222 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.028190    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerID="e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0" exitCode=0
Jul 09 18:38:01.028705 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.028246    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"
Jul 09 18:38:01.028705 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.028275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" event={"ID":"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001","Type":"ContainerDied","Data":"e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0"}
Jul 09 18:38:01.028705 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.028337    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2" event={"ID":"c6acb0f6-b6db-4974-84f9-ad9e6f6a3001","Type":"ContainerDied","Data":"72a9cd479fae52f238ff7ae2259db992bad1369a0a5ea275bc1439e562dae289"}
Jul 09 18:38:01.028705 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.028363    2567 scope.go:117] "RemoveContainer" containerID="1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7"
Jul 09 18:38:01.037295 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.037279    2567 scope.go:117] "RemoveContainer" containerID="e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0"
Jul 09 18:38:01.044808 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.044765    2567 scope.go:117] "RemoveContainer" containerID="1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7"
Jul 09 18:38:01.045037 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:38:01.045017    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7\": container with ID starting with 1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7 not found: ID does not exist" containerID="1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7"
Jul 09 18:38:01.045085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.045045    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7"} err="failed to get container status \"1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7\": rpc error: code = NotFound desc = could not find container \"1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7\": container with ID starting with 1f21050b4b1625333cbec28bb6a3984748a4b2f9895a3cf50b50e775809f87d7 not found: ID does not exist"
Jul 09 18:38:01.045085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.045061    2567 scope.go:117] "RemoveContainer" containerID="e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0"
Jul 09 18:38:01.045287 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:38:01.045272    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0\": container with ID starting with e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0 not found: ID does not exist" containerID="e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0"
Jul 09 18:38:01.045332 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.045291    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0"} err="failed to get container status \"e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0\": rpc error: code = NotFound desc = could not find container \"e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0\": container with ID starting with e715e81e3d52d54789684c0a0253fd1273599f2915b2a04e33c3396f76d117d0 not found: ID does not exist"
Jul 09 18:38:01.049716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.049695    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"]
Jul 09 18:38:01.053315 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:01.053292    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2ff98-predictor-85dc97599c-ldrd2"]
Jul 09 18:38:02.194898 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:02.194865    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" path="/var/lib/kubelet/pods/c6acb0f6-b6db-4974-84f9-ad9e6f6a3001/volumes"
Jul 09 18:38:05.027004 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:05.026973    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:38:05.027460 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:05.027379    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 09 18:38:08.797889 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:08.797854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:08.842957 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:08.842924    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:38:08.859085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:08.859052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:15.027716 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:15.027675    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 09 18:38:20.951706 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:20.951625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:21.163854 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:21.163812    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:25.028352 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:25.028312    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 09 18:38:31.128083 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.128034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:31.228529 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.228474    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"]
Jul 09 18:38:31.229047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.228817    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container" containerID="cri-o://023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45" gracePeriod=30
Jul 09 18:38:31.229047 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.228840    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kube-rbac-proxy" containerID="cri-o://afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5" gracePeriod=30
Jul 09 18:38:31.256368 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.256341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:31.357307 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.357272    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"]
Jul 09 18:38:31.357925 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.357903    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kube-rbac-proxy"
Jul 09 18:38:31.357994 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.357927    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kube-rbac-proxy"
Jul 09 18:38:31.357994 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.357942    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container"
Jul 09 18:38:31.357994 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.357947    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container"
Jul 09 18:38:31.358152 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.358025    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kserve-container"
Jul 09 18:38:31.358152 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.358034    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6acb0f6-b6db-4974-84f9-ad9e6f6a3001" containerName="kube-rbac-proxy"
Jul 09 18:38:31.361228 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.361211    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.363362 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.363338    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-865f9-predictor-serving-cert\""
Jul 09 18:38:31.363485 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.363344    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-865f9-kube-rbac-proxy-sar-config\""
Jul 09 18:38:31.368425 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.368361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"]
Jul 09 18:38:31.407504 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.407417    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:31.470092 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.470056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjqq7\" (UniqueName: \"kubernetes.io/projected/58d06677-b825-4248-93de-56dc204e0c14-kube-api-access-fjqq7\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.470250 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.470100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58d06677-b825-4248-93de-56dc204e0c14-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.470250 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.470127    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58d06677-b825-4248-93de-56dc204e0c14-proxy-tls\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.510549 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.510516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:31.571100 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.571066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjqq7\" (UniqueName: \"kubernetes.io/projected/58d06677-b825-4248-93de-56dc204e0c14-kube-api-access-fjqq7\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.571285 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.571115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58d06677-b825-4248-93de-56dc204e0c14-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.571285 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.571149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58d06677-b825-4248-93de-56dc204e0c14-proxy-tls\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.571837 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.571780    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58d06677-b825-4248-93de-56dc204e0c14-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.573616 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.573596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58d06677-b825-4248-93de-56dc204e0c14-proxy-tls\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.579421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.579396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjqq7\" (UniqueName: \"kubernetes.io/projected/58d06677-b825-4248-93de-56dc204e0c14-kube-api-access-fjqq7\") pod \"success-200-isvc-865f9-predictor-c475ff65f-gcnww\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") " pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.611922 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.611876    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:31.672844 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.672744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:31.808435 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:31.808405    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"]
Jul 09 18:38:31.810178 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:38:31.810151    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d06677_b825_4248_93de_56dc204e0c14.slice/crio-400a759905adc442bf884ba7c563c1ca53df9d12019c1437b9c03c21b26e72da WatchSource:0}: Error finding container 400a759905adc442bf884ba7c563c1ca53df9d12019c1437b9c03c21b26e72da: Status 404 returned error can't find the container with id 400a759905adc442bf884ba7c563c1ca53df9d12019c1437b9c03c21b26e72da
Jul 09 18:38:32.187666 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.187552    2567 generic.go:356] "Generic (PLEG): container finished" podID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerID="afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5" exitCode=2
Jul 09 18:38:32.187666 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.187592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" event={"ID":"f83dc391-abbb-4240-bea5-dc1c707f5925","Type":"ContainerDied","Data":"afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5"}
Jul 09 18:38:32.193119 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.193088    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:38:32.196296 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.196271    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" event={"ID":"58d06677-b825-4248-93de-56dc204e0c14","Type":"ContainerStarted","Data":"4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948"}
Jul 09 18:38:32.196424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.196332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" event={"ID":"58d06677-b825-4248-93de-56dc204e0c14","Type":"ContainerStarted","Data":"4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee"}
Jul 09 18:38:32.196424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.196355    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" event={"ID":"58d06677-b825-4248-93de-56dc204e0c14","Type":"ContainerStarted","Data":"400a759905adc442bf884ba7c563c1ca53df9d12019c1437b9c03c21b26e72da"}
Jul 09 18:38:32.196424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.196379    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:32.196424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.196397    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:32.210008 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:32.209970    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podStartSLOduration=1.209956791 podStartE2EDuration="1.209956791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:38:32.208325444 +0000 UTC m=+2074.555233360" watchObservedRunningTime="2026-07-09 18:38:32.209956791 +0000 UTC m=+2074.556864710"
Jul 09 18:38:33.197069 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:33.197027    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:38:33.837627 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:33.837589    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.45:8643/healthz\": dial tcp 10.132.0.45:8643: connect: connection refused"
Jul 09 18:38:34.201017 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.200981    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:38:34.395818 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.395793    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:38:34.500405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.500315    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k4wv\" (UniqueName: \"kubernetes.io/projected/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-api-access-6k4wv\") pod \"f83dc391-abbb-4240-bea5-dc1c707f5925\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") "
Jul 09 18:38:34.500405 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.500368    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-rbac-proxy-sar-config\") pod \"f83dc391-abbb-4240-bea5-dc1c707f5925\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") "
Jul 09 18:38:34.500590 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.500487    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f83dc391-abbb-4240-bea5-dc1c707f5925-proxy-tls\") pod \"f83dc391-abbb-4240-bea5-dc1c707f5925\" (UID: \"f83dc391-abbb-4240-bea5-dc1c707f5925\") "
Jul 09 18:38:34.500770 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.500737    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f83dc391-abbb-4240-bea5-dc1c707f5925" (UID: "f83dc391-abbb-4240-bea5-dc1c707f5925"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:38:34.500880 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.500837    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:38:34.502519 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.502494    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83dc391-abbb-4240-bea5-dc1c707f5925-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f83dc391-abbb-4240-bea5-dc1c707f5925" (UID: "f83dc391-abbb-4240-bea5-dc1c707f5925"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:38:34.502519 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.502504    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-api-access-6k4wv" (OuterVolumeSpecName: "kube-api-access-6k4wv") pod "f83dc391-abbb-4240-bea5-dc1c707f5925" (UID: "f83dc391-abbb-4240-bea5-dc1c707f5925"). InnerVolumeSpecName "kube-api-access-6k4wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:38:34.601520 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.601486    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f83dc391-abbb-4240-bea5-dc1c707f5925-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:38:34.601674 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:34.601528    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k4wv\" (UniqueName: \"kubernetes.io/projected/f83dc391-abbb-4240-bea5-dc1c707f5925-kube-api-access-6k4wv\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:38:35.028309 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.028268    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 09 18:38:35.207729 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.207694    2567 generic.go:356] "Generic (PLEG): container finished" podID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerID="023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45" exitCode=0
Jul 09 18:38:35.208214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.207734    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" event={"ID":"f83dc391-abbb-4240-bea5-dc1c707f5925","Type":"ContainerDied","Data":"023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45"}
Jul 09 18:38:35.208214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.207750    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"
Jul 09 18:38:35.208214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.207802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg" event={"ID":"f83dc391-abbb-4240-bea5-dc1c707f5925","Type":"ContainerDied","Data":"db20dda74d36ae778bfa3d16543202c6fe80e57fd8128decdda31e0953ce2a2d"}
Jul 09 18:38:35.208214 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.207815    2567 scope.go:117] "RemoveContainer" containerID="afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5"
Jul 09 18:38:35.217324 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.217308    2567 scope.go:117] "RemoveContainer" containerID="023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45"
Jul 09 18:38:35.225256 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.225239    2567 scope.go:117] "RemoveContainer" containerID="afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5"
Jul 09 18:38:35.225519 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:38:35.225502    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5\": container with ID starting with afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5 not found: ID does not exist" containerID="afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5"
Jul 09 18:38:35.225587 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.225530    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5"} err="failed to get container status \"afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5\": rpc error: code = NotFound desc = could not find container \"afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5\": container with ID starting with afd070041b0a0c6b4d1132ed9327007be193c7b3f99ba566fe2d6110d90fdec5 not found: ID does not exist"
Jul 09 18:38:35.225587 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.225552    2567 scope.go:117] "RemoveContainer" containerID="023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45"
Jul 09 18:38:35.225835 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:38:35.225816    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45\": container with ID starting with 023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45 not found: ID does not exist" containerID="023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45"
Jul 09 18:38:35.225883 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.225840    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45"} err="failed to get container status \"023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45\": rpc error: code = NotFound desc = could not find container \"023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45\": container with ID starting with 023293e4c8f882c7556b031b826a9803b160e1d9817d810c3fe07bcf64ecec45 not found: ID does not exist"
Jul 09 18:38:35.229443 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.229421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"]
Jul 09 18:38:35.232347 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:35.232325    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb3ff-predictor-5f8b9cb959-q42kg"]
Jul 09 18:38:36.196398 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:36.196363    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" path="/var/lib/kubelet/pods/f83dc391-abbb-4240-bea5-dc1c707f5925/volumes"
Jul 09 18:38:39.205571 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:39.205543    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:38:39.206122 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:39.206094    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:38:45.028909 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:45.028865    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:38:49.206126 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:49.206085    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:38:57.207805 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:57.207761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:57.548130 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:57.548026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:58.930882 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:58.930750    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:38:58.953561 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:58.953539    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:38:59.206440 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:38:59.206404    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:39:09.206289 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:39:09.206244    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:39:19.206654 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:39:19.206624    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:39:31.444692 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:39:31.444656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:39:31.714856 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:39:31.714747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:39:36.709795 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:39:36.709760    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:41:09.108025 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:41:09.107989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:41:09.194025 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:41:09.193995    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:43:58.962890 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:43:58.962858    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:43:58.986307 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:43:58.986287    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:44:47.316188 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:44:47.316151    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:32.269098 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:45:32.269060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:47:46.156585 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.156484    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:47:46.198652 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.198618    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"]
Jul 09 18:47:46.198888 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.198857    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" containerID="cri-o://4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee" gracePeriod=30
Jul 09 18:47:46.199146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.198905    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kube-rbac-proxy" containerID="cri-o://4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948" gracePeriod=30
Jul 09 18:47:46.222681 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.222651    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:47:46.290712 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.290682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:47:46.918202 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.918165    2567 generic.go:356] "Generic (PLEG): container finished" podID="58d06677-b825-4248-93de-56dc204e0c14" containerID="4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948" exitCode=2
Jul 09 18:47:46.918369 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:46.918238    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" event={"ID":"58d06677-b825-4248-93de-56dc204e0c14","Type":"ContainerDied","Data":"4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948"}
Jul 09 18:47:49.201885 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.201850    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.47:8643/healthz\": dial tcp 10.132.0.47:8643: connect: connection refused"
Jul 09 18:47:49.206175 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.206148    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused"
Jul 09 18:47:49.346770 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.346745    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:47:49.466897 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.466872    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58d06677-b825-4248-93de-56dc204e0c14-kube-rbac-proxy-sar-config\") pod \"58d06677-b825-4248-93de-56dc204e0c14\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") "
Jul 09 18:47:49.467057 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.466903    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58d06677-b825-4248-93de-56dc204e0c14-proxy-tls\") pod \"58d06677-b825-4248-93de-56dc204e0c14\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") "
Jul 09 18:47:49.467057 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.467007    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjqq7\" (UniqueName: \"kubernetes.io/projected/58d06677-b825-4248-93de-56dc204e0c14-kube-api-access-fjqq7\") pod \"58d06677-b825-4248-93de-56dc204e0c14\" (UID: \"58d06677-b825-4248-93de-56dc204e0c14\") "
Jul 09 18:47:49.467263 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.467236    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d06677-b825-4248-93de-56dc204e0c14-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "58d06677-b825-4248-93de-56dc204e0c14" (UID: "58d06677-b825-4248-93de-56dc204e0c14"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:47:49.469061 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.469031    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d06677-b825-4248-93de-56dc204e0c14-kube-api-access-fjqq7" (OuterVolumeSpecName: "kube-api-access-fjqq7") pod "58d06677-b825-4248-93de-56dc204e0c14" (UID: "58d06677-b825-4248-93de-56dc204e0c14"). InnerVolumeSpecName "kube-api-access-fjqq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:47:49.469161 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.469110    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d06677-b825-4248-93de-56dc204e0c14-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "58d06677-b825-4248-93de-56dc204e0c14" (UID: "58d06677-b825-4248-93de-56dc204e0c14"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:47:49.567741 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.567706    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjqq7\" (UniqueName: \"kubernetes.io/projected/58d06677-b825-4248-93de-56dc204e0c14-kube-api-access-fjqq7\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:47:49.567741 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.567734    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58d06677-b825-4248-93de-56dc204e0c14-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:47:49.567741 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.567746    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58d06677-b825-4248-93de-56dc204e0c14-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:47:49.933143 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.933052    2567 generic.go:356] "Generic (PLEG): container finished" podID="58d06677-b825-4248-93de-56dc204e0c14" containerID="4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee" exitCode=0
Jul 09 18:47:49.933143 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.933097    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"
Jul 09 18:47:49.933386 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.933142    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" event={"ID":"58d06677-b825-4248-93de-56dc204e0c14","Type":"ContainerDied","Data":"4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee"}
Jul 09 18:47:49.933386 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.933210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww" event={"ID":"58d06677-b825-4248-93de-56dc204e0c14","Type":"ContainerDied","Data":"400a759905adc442bf884ba7c563c1ca53df9d12019c1437b9c03c21b26e72da"}
Jul 09 18:47:49.933386 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.933254    2567 scope.go:117] "RemoveContainer" containerID="4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948"
Jul 09 18:47:49.942443 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.942420    2567 scope.go:117] "RemoveContainer" containerID="4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee"
Jul 09 18:47:49.949954 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.949938    2567 scope.go:117] "RemoveContainer" containerID="4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948"
Jul 09 18:47:49.950232 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:47:49.950213    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948\": container with ID starting with 4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948 not found: ID does not exist" containerID="4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948"
Jul 09 18:47:49.950280 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.950241    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948"} err="failed to get container status \"4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948\": rpc error: code = NotFound desc = could not find container \"4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948\": container with ID starting with 4a648e56ba94be13e91d383cff2b450a574fb59e379cba50f98ec021caa23948 not found: ID does not exist"
Jul 09 18:47:49.950280 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.950258    2567 scope.go:117] "RemoveContainer" containerID="4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee"
Jul 09 18:47:49.950479 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:47:49.950458    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee\": container with ID starting with 4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee not found: ID does not exist" containerID="4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee"
Jul 09 18:47:49.950524 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.950485    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee"} err="failed to get container status \"4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee\": rpc error: code = NotFound desc = could not find container \"4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee\": container with ID starting with 4247a8fc75d3766e413b77899d8661e16f8207bf5e54a1f4f8dec311845639ee not found: ID does not exist"
Jul 09 18:47:49.955193 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.955173    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"]
Jul 09 18:47:49.959161 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:49.959138    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-865f9-predictor-c475ff65f-gcnww"]
Jul 09 18:47:50.195226 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:47:50.195195    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d06677-b825-4248-93de-56dc204e0c14" path="/var/lib/kubelet/pods/58d06677-b825-4248-93de-56dc204e0c14/volumes"
Jul 09 18:48:58.996690 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:48:58.996581    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:48:59.018401 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:48:59.018374    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:51:13.321833 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:51:13.321778    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:53:04.469051 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:53:04.469015    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:53:59.028921 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:53:59.028776    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:53:59.049722 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:53:59.049700    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:55:16.365761 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:16.365667    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:16.461304 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:16.461270    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"]
Jul 09 18:55:16.461558 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:16.461533    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container" containerID="cri-o://5f2eda0c06468906d56150fb84e13cf6df8c9e6c952f3bb98b839152cd4b88bd" gracePeriod=30
Jul 09 18:55:16.461622 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:16.461562    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kube-rbac-proxy" containerID="cri-o://75b1818bdb0d8e237711d7299c9d150193bdb2c33c2ad8e694b1a12ac2617e38" gracePeriod=30
Jul 09 18:55:16.488132 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:16.488098    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:16.567462 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:16.567414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:17.100293 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.100258    2567 generic.go:356] "Generic (PLEG): container finished" podID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerID="75b1818bdb0d8e237711d7299c9d150193bdb2c33c2ad8e694b1a12ac2617e38" exitCode=2
Jul 09 18:55:17.100475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.100315    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" event={"ID":"390d1d1c-ae2a-4732-95a8-107dda4f14df","Type":"ContainerDied","Data":"75b1818bdb0d8e237711d7299c9d150193bdb2c33c2ad8e694b1a12ac2617e38"}
Jul 09 18:55:17.163870 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.163835    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kldf4/must-gather-hcjxt"]
Jul 09 18:55:17.164227 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164215    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kube-rbac-proxy"
Jul 09 18:55:17.164270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164228    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kube-rbac-proxy"
Jul 09 18:55:17.164270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164242    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kube-rbac-proxy"
Jul 09 18:55:17.164270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164248    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kube-rbac-proxy"
Jul 09 18:55:17.164270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164261    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container"
Jul 09 18:55:17.164270 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164267    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container"
Jul 09 18:55:17.164421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164274    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container"
Jul 09 18:55:17.164421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164279    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container"
Jul 09 18:55:17.164421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164331    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kube-rbac-proxy"
Jul 09 18:55:17.164421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164344    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kube-rbac-proxy"
Jul 09 18:55:17.164421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164351    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d06677-b825-4248-93de-56dc204e0c14" containerName="kserve-container"
Jul 09 18:55:17.164421 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.164356    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83dc391-abbb-4240-bea5-dc1c707f5925" containerName="kserve-container"
Jul 09 18:55:17.167450 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.167426    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.169917 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.169891    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kldf4\"/\"kube-root-ca.crt\""
Jul 09 18:55:17.169917 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.169913    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kldf4\"/\"openshift-service-ca.crt\""
Jul 09 18:55:17.170081 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.169973    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kldf4\"/\"default-dockercfg-phzlk\""
Jul 09 18:55:17.177207 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.177187    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kldf4/must-gather-hcjxt"]
Jul 09 18:55:17.181865 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.181842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:17.240433 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.240395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52wbv\" (UniqueName: \"kubernetes.io/projected/6e0fbd28-2aed-4c9a-b19a-9e1069816307-kube-api-access-52wbv\") pod \"must-gather-hcjxt\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") " pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.240617 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.240481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e0fbd28-2aed-4c9a-b19a-9e1069816307-must-gather-output\") pod \"must-gather-hcjxt\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") " pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.245423 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.245393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:17.341275 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.341234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52wbv\" (UniqueName: \"kubernetes.io/projected/6e0fbd28-2aed-4c9a-b19a-9e1069816307-kube-api-access-52wbv\") pod \"must-gather-hcjxt\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") " pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.341443 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.341282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e0fbd28-2aed-4c9a-b19a-9e1069816307-must-gather-output\") pod \"must-gather-hcjxt\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") " pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.341618 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.341602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e0fbd28-2aed-4c9a-b19a-9e1069816307-must-gather-output\") pod \"must-gather-hcjxt\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") " pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.353573 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.350000    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52wbv\" (UniqueName: \"kubernetes.io/projected/6e0fbd28-2aed-4c9a-b19a-9e1069816307-kube-api-access-52wbv\") pod \"must-gather-hcjxt\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") " pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.488878 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.488843    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:17.624554 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.624527    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kldf4/must-gather-hcjxt"]
Jul 09 18:55:17.626745 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:55:17.626720    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e0fbd28_2aed_4c9a_b19a_9e1069816307.slice/crio-06f780a62f04356cd74a6dfe2c27a443ccb66b6c8004685fe7ead5a64567caa5 WatchSource:0}: Error finding container 06f780a62f04356cd74a6dfe2c27a443ccb66b6c8004685fe7ead5a64567caa5: Status 404 returned error can't find the container with id 06f780a62f04356cd74a6dfe2c27a443ccb66b6c8004685fe7ead5a64567caa5
Jul 09 18:55:17.628514 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:17.628496    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:55:18.106806 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:18.106743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kldf4/must-gather-hcjxt" event={"ID":"6e0fbd28-2aed-4c9a-b19a-9e1069816307","Type":"ContainerStarted","Data":"06f780a62f04356cd74a6dfe2c27a443ccb66b6c8004685fe7ead5a64567caa5"}
Jul 09 18:55:20.022506 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:20.022460    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.46:8643/healthz\": dial tcp 10.132.0.46:8643: connect: connection refused"
Jul 09 18:55:20.122814 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:20.122708    2567 generic.go:356] "Generic (PLEG): container finished" podID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerID="5f2eda0c06468906d56150fb84e13cf6df8c9e6c952f3bb98b839152cd4b88bd" exitCode=0
Jul 09 18:55:20.122814 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:20.122772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" event={"ID":"390d1d1c-ae2a-4732-95a8-107dda4f14df","Type":"ContainerDied","Data":"5f2eda0c06468906d56150fb84e13cf6df8c9e6c952f3bb98b839152cd4b88bd"}
Jul 09 18:55:23.633314 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.633285    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:55:23.702156 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.702119    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-rbac-proxy-sar-config\") pod \"390d1d1c-ae2a-4732-95a8-107dda4f14df\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") "
Jul 09 18:55:23.702342 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.702179    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmljj\" (UniqueName: \"kubernetes.io/projected/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-api-access-pmljj\") pod \"390d1d1c-ae2a-4732-95a8-107dda4f14df\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") "
Jul 09 18:55:23.702342 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.702264    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls\") pod \"390d1d1c-ae2a-4732-95a8-107dda4f14df\" (UID: \"390d1d1c-ae2a-4732-95a8-107dda4f14df\") "
Jul 09 18:55:23.702543 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.702515    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "390d1d1c-ae2a-4732-95a8-107dda4f14df" (UID: "390d1d1c-ae2a-4732-95a8-107dda4f14df"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:55:23.704538 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.704512    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "390d1d1c-ae2a-4732-95a8-107dda4f14df" (UID: "390d1d1c-ae2a-4732-95a8-107dda4f14df"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:55:23.704777 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.704753    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-api-access-pmljj" (OuterVolumeSpecName: "kube-api-access-pmljj") pod "390d1d1c-ae2a-4732-95a8-107dda4f14df" (UID: "390d1d1c-ae2a-4732-95a8-107dda4f14df"). InnerVolumeSpecName "kube-api-access-pmljj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:55:23.803204 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.803174    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:55:23.803204 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.803203    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmljj\" (UniqueName: \"kubernetes.io/projected/390d1d1c-ae2a-4732-95a8-107dda4f14df-kube-api-access-pmljj\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:55:23.803388 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:23.803217    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/390d1d1c-ae2a-4732-95a8-107dda4f14df-proxy-tls\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:55:24.148393 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.148355    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kldf4/must-gather-hcjxt" event={"ID":"6e0fbd28-2aed-4c9a-b19a-9e1069816307","Type":"ContainerStarted","Data":"23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"}
Jul 09 18:55:24.150467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.150238    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh" event={"ID":"390d1d1c-ae2a-4732-95a8-107dda4f14df","Type":"ContainerDied","Data":"553ae34d5e4acdbedde0b6d9dec87679d4d73d7ea72fceb4b0be39681c205b5e"}
Jul 09 18:55:24.150467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.150338    2567 scope.go:117] "RemoveContainer" containerID="75b1818bdb0d8e237711d7299c9d150193bdb2c33c2ad8e694b1a12ac2617e38"
Jul 09 18:55:24.150467 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.150374    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"
Jul 09 18:55:24.159769 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.159745    2567 scope.go:117] "RemoveContainer" containerID="5f2eda0c06468906d56150fb84e13cf6df8c9e6c952f3bb98b839152cd4b88bd"
Jul 09 18:55:24.175826 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.175680    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"]
Jul 09 18:55:24.181053 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.181025    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bed12-predictor-6d68b6c7b7-ldlsh"]
Jul 09 18:55:24.195953 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:24.195927    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" path="/var/lib/kubelet/pods/390d1d1c-ae2a-4732-95a8-107dda4f14df/volumes"
Jul 09 18:55:25.158689 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:25.158640    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kldf4/must-gather-hcjxt" event={"ID":"6e0fbd28-2aed-4c9a-b19a-9e1069816307","Type":"ContainerStarted","Data":"fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9"}
Jul 09 18:55:46.270450 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:46.270362    2567 generic.go:356] "Generic (PLEG): container finished" podID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerID="23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea" exitCode=0
Jul 09 18:55:46.270982 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:46.270444    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kldf4/must-gather-hcjxt" event={"ID":"6e0fbd28-2aed-4c9a-b19a-9e1069816307","Type":"ContainerDied","Data":"23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"}
Jul 09 18:55:46.270982 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:46.270850    2567 scope.go:117] "RemoveContainer" containerID="23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"
Jul 09 18:55:46.978727 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:46.978695    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kldf4_must-gather-hcjxt_6e0fbd28-2aed-4c9a-b19a-9e1069816307/gather/0.log"
Jul 09 18:55:47.463651 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:47.463606    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:47.624199 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:47.624163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:47.702828 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:47.702800    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:50.342046 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:50.342012    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sbqht_41c442b1-6673-47d3-bc8f-38abe3b59588/global-pull-secret-syncer/0.log"
Jul 09 18:55:50.452952 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:50.452921    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log"
Jul 09 18:55:50.554959 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:50.554929    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-102.ec2.internal_33d903d9ca629bd1f13fc91f98e8c89a/haproxy/0.log"
Jul 09 18:55:52.533596 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.533560    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-kldf4/must-gather-hcjxt"]
Jul 09 18:55:52.534132 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.533729    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-kldf4/must-gather-hcjxt" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="copy" containerID="cri-o://fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9" gracePeriod=2
Jul 09 18:55:52.539591 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.539558    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-kldf4/must-gather-hcjxt"]
Jul 09 18:55:52.766362 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.766342    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kldf4_must-gather-hcjxt_6e0fbd28-2aed-4c9a-b19a-9e1069816307/copy/0.log"
Jul 09 18:55:52.767940 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.767919    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:52.769995 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:55:52.769971    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hcjxt\" is forbidden: User \"system:node:ip-10-0-132-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kldf4\": no relationship found between node 'ip-10-0-132-102.ec2.internal' and this object" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:52.859608 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.859521    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e0fbd28-2aed-4c9a-b19a-9e1069816307-must-gather-output\") pod \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") "
Jul 09 18:55:52.859608 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.859566    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52wbv\" (UniqueName: \"kubernetes.io/projected/6e0fbd28-2aed-4c9a-b19a-9e1069816307-kube-api-access-52wbv\") pod \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\" (UID: \"6e0fbd28-2aed-4c9a-b19a-9e1069816307\") "
Jul 09 18:55:52.861064 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.861040    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e0fbd28-2aed-4c9a-b19a-9e1069816307-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "6e0fbd28-2aed-4c9a-b19a-9e1069816307" (UID: "6e0fbd28-2aed-4c9a-b19a-9e1069816307"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:55:52.861747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.861723    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e0fbd28-2aed-4c9a-b19a-9e1069816307-kube-api-access-52wbv" (OuterVolumeSpecName: "kube-api-access-52wbv") pod "6e0fbd28-2aed-4c9a-b19a-9e1069816307" (UID: "6e0fbd28-2aed-4c9a-b19a-9e1069816307"). InnerVolumeSpecName "kube-api-access-52wbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:55:52.960475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.960444    2567 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e0fbd28-2aed-4c9a-b19a-9e1069816307-must-gather-output\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:55:52.960475 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:52.960471    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52wbv\" (UniqueName: \"kubernetes.io/projected/6e0fbd28-2aed-4c9a-b19a-9e1069816307-kube-api-access-52wbv\") on node \"ip-10-0-132-102.ec2.internal\" DevicePath \"\""
Jul 09 18:55:53.305108 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.305081    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kldf4_must-gather-hcjxt_6e0fbd28-2aed-4c9a-b19a-9e1069816307/copy/0.log"
Jul 09 18:55:53.305466 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.305442    2567 generic.go:356] "Generic (PLEG): container finished" podID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerID="fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9" exitCode=143
Jul 09 18:55:53.305526 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.305497    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:53.305526 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.305509    2567 scope.go:117] "RemoveContainer" containerID="fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9"
Jul 09 18:55:53.308015 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:55:53.307990    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hcjxt\" is forbidden: User \"system:node:ip-10-0-132-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kldf4\": no relationship found between node 'ip-10-0-132-102.ec2.internal' and this object" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:53.314756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.314735    2567 scope.go:117] "RemoveContainer" containerID="23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"
Jul 09 18:55:53.316905 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:55:53.316870    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hcjxt\" is forbidden: User \"system:node:ip-10-0-132-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kldf4\": no relationship found between node 'ip-10-0-132-102.ec2.internal' and this object" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" pod="openshift-must-gather-kldf4/must-gather-hcjxt"
Jul 09 18:55:53.317511 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.317490    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:53.326796 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.326763    2567 scope.go:117] "RemoveContainer" containerID="fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9"
Jul 09 18:55:53.327053 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:55:53.327036    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9\": container with ID starting with fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9 not found: ID does not exist" containerID="fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9"
Jul 09 18:55:53.327119 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.327059    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9"} err="failed to get container status \"fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9\": rpc error: code = NotFound desc = could not find container \"fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9\": container with ID starting with fab1d86e451613d10988ef3bf6130c3b09d43409436af553c3a5cedb003beac9 not found: ID does not exist"
Jul 09 18:55:53.327119 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.327075    2567 scope.go:117] "RemoveContainer" containerID="23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"
Jul 09 18:55:53.327316 ip-10-0-132-102 kubenswrapper[2567]: E0709 18:55:53.327297    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea\": container with ID starting with 23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea not found: ID does not exist" containerID="23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"
Jul 09 18:55:53.327372 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.327321    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea"} err="failed to get container status \"23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea\": rpc error: code = NotFound desc = could not find container \"23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea\": container with ID starting with 23df248c149d8094eff5c67d80642a00bcb8c84aa05c256097b525a0255c0cea not found: ID does not exist"
Jul 09 18:55:53.482138 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:53.482107    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrz6_965f1315-c272-4861-9ab0-a1f343b6414f/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:54.195253 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.195215    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" path="/var/lib/kubelet/pods/6e0fbd28-2aed-4c9a-b19a-9e1069816307/volumes"
Jul 09 18:55:54.598552 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.598470    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/alertmanager/0.log"
Jul 09 18:55:54.625373 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.625347    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/config-reloader/0.log"
Jul 09 18:55:54.647470 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.647442    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/kube-rbac-proxy-web/0.log"
Jul 09 18:55:54.674007 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.673982    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/kube-rbac-proxy/0.log"
Jul 09 18:55:54.700060 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.700038    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/kube-rbac-proxy-metric/0.log"
Jul 09 18:55:54.724756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.724737    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/prom-label-proxy/0.log"
Jul 09 18:55:54.754521 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.754494    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ca9a665-cfcc-4541-8041-fd5822781959/init-config-reloader/0.log"
Jul 09 18:55:54.807141 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.807112    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-2qkks_e6c97669-d87d-4e2f-b35c-c7a39853020c/cluster-monitoring-operator/0.log"
Jul 09 18:55:54.921124 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:54.921044    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58c7786f4b-k57cq_203d4f89-7850-46c7-952d-c6c8c0730ec0/metrics-server/0.log"
Jul 09 18:55:55.032122 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.032091    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-47fhv_114cd342-a369-4587-bf22-786756baa888/node-exporter/0.log"
Jul 09 18:55:55.063278 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.063248    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-47fhv_114cd342-a369-4587-bf22-786756baa888/kube-rbac-proxy/0.log"
Jul 09 18:55:55.086581 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.086557    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-47fhv_114cd342-a369-4587-bf22-786756baa888/init-textfile/0.log"
Jul 09 18:55:55.280070 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.280041    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s68lq_21121af4-c2c8-4f75-880a-b33e78df782b/kube-rbac-proxy-main/0.log"
Jul 09 18:55:55.300428 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.300400    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s68lq_21121af4-c2c8-4f75-880a-b33e78df782b/kube-rbac-proxy-self/0.log"
Jul 09 18:55:55.328517 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.328484    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s68lq_21121af4-c2c8-4f75-880a-b33e78df782b/openshift-state-metrics/0.log"
Jul 09 18:55:55.530616 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.530531    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-c59fq_5c0c272e-4bcc-4608-b033-7413930c4dda/prometheus-operator/0.log"
Jul 09 18:55:55.546767 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.546734    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-c59fq_5c0c272e-4bcc-4608-b033-7413930c4dda/kube-rbac-proxy/0.log"
Jul 09 18:55:55.572160 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.572136    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-xxt4p_3d0b072b-ad0d-450a-a008-1f4094360d95/prometheus-operator-admission-webhook/0.log"
Jul 09 18:55:55.607424 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.607391    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66dbd5c75-zxkf9_ffb23c53-46ab-47e3-b760-f5bd4e30019f/telemeter-client/0.log"
Jul 09 18:55:55.630149 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.630120    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66dbd5c75-zxkf9_ffb23c53-46ab-47e3-b760-f5bd4e30019f/reload/0.log"
Jul 09 18:55:55.654462 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.654436    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66dbd5c75-zxkf9_ffb23c53-46ab-47e3-b760-f5bd4e30019f/kube-rbac-proxy/0.log"
Jul 09 18:55:55.691205 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.691173    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/thanos-query/0.log"
Jul 09 18:55:55.713069 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.713044    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy-web/0.log"
Jul 09 18:55:55.736338 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.736303    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy/0.log"
Jul 09 18:55:55.759837 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.759810    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/prom-label-proxy/0.log"
Jul 09 18:55:55.780812 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.780715    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy-rules/0.log"
Jul 09 18:55:55.805790 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:55.805765    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b79df75c8-lwp5k_029ed26f-317b-4f04-88d0-3a72273a8c87/kube-rbac-proxy-metrics/0.log"
Jul 09 18:55:57.099687 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:57.099657    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-hgdbm_8bbaae80-06b4-43bf-92f1-def9f70426f4/networking-console-plugin/0.log"
Jul 09 18:55:58.040658 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.040625    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fk6jw_e30217fd-44c2-41e6-b154-6e9d2e35ce6f/download-server/0.log"
Jul 09 18:55:58.539091 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.539066    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-2hvxp_d09d07a2-835f-49b3-a92b-98d1ab552714/volume-data-source-validator/0.log"
Jul 09 18:55:58.696011 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.695978    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"]
Jul 09 18:55:58.696368 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696356    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kube-rbac-proxy"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696369    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kube-rbac-proxy"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696379    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="gather"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696384    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="gather"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696395    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696401    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696407    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="copy"
Jul 09 18:55:58.696414 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696413    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="copy"
Jul 09 18:55:58.696648 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696462    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kube-rbac-proxy"
Jul 09 18:55:58.696648 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696471    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="390d1d1c-ae2a-4732-95a8-107dda4f14df" containerName="kserve-container"
Jul 09 18:55:58.696648 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696478    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="copy"
Jul 09 18:55:58.696648 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.696485    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e0fbd28-2aed-4c9a-b19a-9e1069816307" containerName="gather"
Jul 09 18:55:58.700134 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.700115    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.702963 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.702940    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8wgtz\"/\"default-dockercfg-k5z4r\""
Jul 09 18:55:58.703090 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.702942    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8wgtz\"/\"kube-root-ca.crt\""
Jul 09 18:55:58.703609 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.703594    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8wgtz\"/\"openshift-service-ca.crt\""
Jul 09 18:55:58.709109 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.709085    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"]
Jul 09 18:55:58.811085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.811000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2s55\" (UniqueName: \"kubernetes.io/projected/64f37088-46f0-4c43-a9d9-37ed3981e72b-kube-api-access-n2s55\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.811085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.811042    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-sys\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.811085 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.811067    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-podres\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.811307 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.811162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-proc\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.811307 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.811205    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-lib-modules\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912494 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912455    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2s55\" (UniqueName: \"kubernetes.io/projected/64f37088-46f0-4c43-a9d9-37ed3981e72b-kube-api-access-n2s55\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-sys\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-podres\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912578    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-proc\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912588    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-sys\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-lib-modules\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.912694 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912670    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-proc\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.913003 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912723    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-podres\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.913003 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.912833    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64f37088-46f0-4c43-a9d9-37ed3981e72b-lib-modules\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:58.920430 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:58.920411    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2s55\" (UniqueName: \"kubernetes.io/projected/64f37088-46f0-4c43-a9d9-37ed3981e72b-kube-api-access-n2s55\") pod \"perf-node-gather-daemonset-whk76\" (UID: \"64f37088-46f0-4c43-a9d9-37ed3981e72b\") " pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:59.010853 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.010814    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:59.146092 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.146064    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"]
Jul 09 18:55:59.149152 ip-10-0-132-102 kubenswrapper[2567]: W0709 18:55:59.149122    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod64f37088_46f0_4c43_a9d9_37ed3981e72b.slice/crio-beb8c8ba1b0bd9a75a37519ceea2b1589a38343d39de4e19c6f87a1d17d6fbef WatchSource:0}: Error finding container beb8c8ba1b0bd9a75a37519ceea2b1589a38343d39de4e19c6f87a1d17d6fbef: Status 404 returned error can't find the container with id beb8c8ba1b0bd9a75a37519ceea2b1589a38343d39de4e19c6f87a1d17d6fbef
Jul 09 18:55:59.336392 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.336305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76" event={"ID":"64f37088-46f0-4c43-a9d9-37ed3981e72b","Type":"ContainerStarted","Data":"fea520232a577ae7483ba491925a49e8140e2fe40f71ff503c7a40fa3dde9fa4"}
Jul 09 18:55:59.336392 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.336356    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76" event={"ID":"64f37088-46f0-4c43-a9d9-37ed3981e72b","Type":"ContainerStarted","Data":"beb8c8ba1b0bd9a75a37519ceea2b1589a38343d39de4e19c6f87a1d17d6fbef"}
Jul 09 18:55:59.336588 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.336409    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:55:59.353752 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.353707    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76" podStartSLOduration=1.353694118 podStartE2EDuration="1.353694118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:55:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:59.353616765 +0000 UTC m=+3121.700524681" watchObservedRunningTime="2026-07-09 18:55:59.353694118 +0000 UTC m=+3121.700602031"
Jul 09 18:55:59.568228 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.568200    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lctck_cb03f821-623c-4a50-841d-8642fa93b2ef/dns/0.log"
Jul 09 18:55:59.587415 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.587342    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lctck_cb03f821-623c-4a50-841d-8642fa93b2ef/kube-rbac-proxy/0.log"
Jul 09 18:55:59.633406 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:55:59.633381    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rcdz4_cc25caa5-2d30-450f-958e-0c8b51386e5c/dns-node-resolver/0.log"
Jul 09 18:56:00.183999 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:00.183967    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9lw8l_dd6e4bba-e65a-4097-9a87-4d6ca3ac163a/node-ca/0.log"
Jul 09 18:56:01.397980 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:01.397951    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hnmt_0236ff24-a897-4d0f-b0b0-23d3a7904f2b/serve-healthcheck-canary/0.log"
Jul 09 18:56:01.821763 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:01.821724    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-lplzp_a189a918-de0e-49b0-a269-79ef6f699c06/insights-operator/0.log"
Jul 09 18:56:01.825745 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:01.825718    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-lplzp_a189a918-de0e-49b0-a269-79ef6f699c06/insights-operator/1.log"
Jul 09 18:56:01.914872 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:01.914840    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rjxn2_e2efa792-8624-4922-a9ca-9617fdb18a02/kube-rbac-proxy/0.log"
Jul 09 18:56:01.933827 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:01.933775    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rjxn2_e2efa792-8624-4922-a9ca-9617fdb18a02/exporter/0.log"
Jul 09 18:56:01.953995 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:01.953973    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rjxn2_e2efa792-8624-4922-a9ca-9617fdb18a02/extractor/0.log"
Jul 09 18:56:04.250252 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:04.250222    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-hmptm_485647a0-d9fe-4a40-b666-3780ed978ce1/manager/0.log"
Jul 09 18:56:04.292146 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:04.292115    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-mq4qw_757be998-2ce7-4d10-ac7c-b16df8cf728e/s3-init/0.log"
Jul 09 18:56:04.307492 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:04.307471    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-mq4qw_757be998-2ce7-4d10-ac7c-b16df8cf728e/download-hf-model/0.log"
Jul 09 18:56:04.336538 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:04.336514    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-92bdr_35afea2b-5da4-4ae8-9157-9cfb0cd9c9b1/seaweedfs/0.log"
Jul 09 18:56:05.351910 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:05.351883    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8wgtz/perf-node-gather-daemonset-whk76"
Jul 09 18:56:08.117286 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:08.117250    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-k2rs2_08575775-9374-4700-adbc-2a111c725eec/migrator/0.log"
Jul 09 18:56:08.138756 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:08.138723    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-k2rs2_08575775-9374-4700-adbc-2a111c725eec/graceful-termination/0.log"
Jul 09 18:56:08.508804 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:08.508755    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-q7z4l_9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc/kube-storage-version-migrator-operator/1.log"
Jul 09 18:56:08.510372 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:08.510347    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-q7z4l_9cf0633c-0a0f-4937-91f9-b9adfcbf5ecc/kube-storage-version-migrator-operator/0.log"
Jul 09 18:56:09.608956 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:09.608929    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8mh85_e0a9c2d9-dc5c-48a1-86f5-1267d0c91227/kube-multus/0.log"
Jul 09 18:56:10.080683 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.080605    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/kube-multus-additional-cni-plugins/0.log"
Jul 09 18:56:10.121546 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.121520    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/egress-router-binary-copy/0.log"
Jul 09 18:56:10.164566 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.164538    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/cni-plugins/0.log"
Jul 09 18:56:10.200014 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.199985    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/bond-cni-plugin/0.log"
Jul 09 18:56:10.232436 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.232409    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/routeoverride-cni/0.log"
Jul 09 18:56:10.269854 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.269827    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/whereabouts-cni-bincopy/0.log"
Jul 09 18:56:10.297583 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.297556    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rxs4t_5daf7f63-4e19-423d-9361-06c07df0e551/whereabouts-cni/0.log"
Jul 09 18:56:10.480246 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.480214    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nm5xr_bcfc3b54-df61-4210-b823-da5675e01562/network-metrics-daemon/0.log"
Jul 09 18:56:10.506331 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:10.506301    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nm5xr_bcfc3b54-df61-4210-b823-da5675e01562/kube-rbac-proxy/0.log"
Jul 09 18:56:11.724482 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.724447    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-controller/0.log"
Jul 09 18:56:11.741604 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.741572    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/1.log"
Jul 09 18:56:11.741604 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.741598    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovn-acl-logging/0.log"
Jul 09 18:56:11.764221 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.764189    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/kube-rbac-proxy-node/0.log"
Jul 09 18:56:11.789857 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.789829    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/kube-rbac-proxy-ovn-metrics/0.log"
Jul 09 18:56:11.806051 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.806027    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/northd/0.log"
Jul 09 18:56:11.827891 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.827858    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/nbdb/0.log"
Jul 09 18:56:11.850012 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.849984    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/sbdb/0.log"
Jul 09 18:56:11.915237 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:11.915209    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rpvm4_42ad5776-8b11-47ee-a89a-b9fc896d5a49/ovnkube-controller/0.log"
Jul 09 18:56:12.969864 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:12.969833    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-wpfsc_28fe1fbc-df12-44ba-ab2c-f8ba3c306afe/network-check-target-container/0.log"
Jul 09 18:56:14.028205 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:14.028177    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-w6gpm_d58ddb01-ee8d-420a-8b95-96d197946d59/iptables-alerter/0.log"
Jul 09 18:56:14.645747 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:14.645717    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-gg6m4_07b80820-91f7-432a-981a-5b6f9b0792e0/tuned/0.log"
Jul 09 18:56:16.534725 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:16.534696    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-65db858894-fzdlj_f9296204-be27-4bae-8c31-3c88a9c08433/cluster-samples-operator/0.log"
Jul 09 18:56:16.551690 ip-10-0-132-102 kubenswrapper[2567]: I0709 18:56:16.551666    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-65db858894-fzdlj_f9296204-be27-4bae-8c31-3c88a9c08433/cluster-samples-operator-watch/0.log"