Jul 13 18:48:21.354476 ip-10-0-134-48 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 18:48:21.354487 ip-10-0-134-48 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 18:48:21.354496 ip-10-0-134-48 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 18:48:21.354806 ip-10-0-134-48 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 18:48:31.503379 ip-10-0-134-48 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 18:48:31.503396 ip-10-0-134-48 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e5b822e932a143f5bfaccd36a9140fde --
Jul 13 18:50:30.834842 ip-10-0-134-48 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 18:50:31.242617 ip-10-0-134-48 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 18:50:31.242617 ip-10-0-134-48 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 18:50:31.242617 ip-10-0-134-48 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 18:50:31.242617 ip-10-0-134-48 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 18:50:31.242617 ip-10-0-134-48 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 18:50:31.243490 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.243405    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 18:50:31.248190 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248174    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:50:31.248190 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248190    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248194    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248197    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248200    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248203    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248205    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248211    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248215    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248218    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248220    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248223    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248226    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248230    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248232    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248235    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248237    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248241    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248243    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248246    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:50:31.248260 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248248    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248251    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248254    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248257    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248259    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248262    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248264    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248267    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248270    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248273    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248276    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248280    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248284    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248287    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248290    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248293    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248295    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248297    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248300    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:50:31.248732 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248302    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248305    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248308    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248310    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248313    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248316    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248318    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248321    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248323    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248326    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248328    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248330    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248333    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248336    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248339    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248343    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248345    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248348    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248350    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248353    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:50:31.249194 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248356    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248358    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248360    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248363    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248365    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248368    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248370    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248373    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248375    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248378    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248380    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248383    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248385    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248388    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248391    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248393    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248396    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248398    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248401    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248403    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:50:31.249694 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248406    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248408    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248411    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248413    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248417    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248421    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248425    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248427    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248430    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248433    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248437    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248439    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248442    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248444    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248447    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248450    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248453    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248455    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248458    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:50:31.250157 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.248460    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:50:31.250683 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250671    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:50:31.250683 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250682    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250686    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250689    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250692    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250694    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250698    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250701    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250705    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250708    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250711    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250715    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250718    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250721    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250723    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250726    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250728    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250731    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250734    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250736    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250739    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:50:31.250736 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250742    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250745    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250748    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250750    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250753    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250755    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250758    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250762    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250765    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250768    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250770    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250773    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250776    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250779    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250781    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250784    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250786    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250789    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250791    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250794    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:50:31.251256 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250797    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250799    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250802    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250810    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250813    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250815    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250818    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250820    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250823    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250825    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250828    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250830    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250832    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250835    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250838    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250841    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250844    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250846    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250849    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250851    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:50:31.251827 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250854    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250857    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250860    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250862    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250865    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250868    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250870    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250873    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250875    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250878    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250880    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250883    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250885    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250890    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250894    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250896    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250899    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250902    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250905    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250908    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:50:31.252312 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250910    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250912    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250915    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250917    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250920    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250924    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250928    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250931    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250934    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250937    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250940    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250942    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250945    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250947    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250950    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250953    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250955    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.250958    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251337    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:50:31.252790 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251341    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251344    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251347    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251350    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251352    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251355    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251358    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251360    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251363    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251372    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251375    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251378    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251381    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251384    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251386    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251388    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251391    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251393    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251396    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251399    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:50:31.253224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251401    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251404    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251408    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251412    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251415    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251418    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251421    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251423    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251427    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251430    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251432    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251435    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251438    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251441    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251443    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251445    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251448    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251450    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251453    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:50:31.253713 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251455    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251457    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251460    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251466    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251468    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251471    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251473    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251476    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251478    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251480    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251483    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251486    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251489    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251492    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251494    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251497    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251500    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251502    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251505    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251507    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:50:31.254180 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251510    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251512    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251515    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251517    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251519    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251522    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251524    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251527    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251529    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251532    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251534    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251536    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251540    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251543    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251545    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251549    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251570    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251573    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251577    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251579    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:50:31.254672 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251582    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251584    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251587    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251589    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251592    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251594    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251597    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251599    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251602    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251605    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251607    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251610    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251612    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251615    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251617    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251621    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251625    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251628    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.251631    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251703    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251714    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 18:50:31.255148 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251720    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251724    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251728    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251732    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251735    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251742    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251752    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251755    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251760    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251763    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251766    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251770    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251773    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251776    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251779    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251782    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251785    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251788    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251791    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251794    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251797    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251801    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251804    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 18:50:31.255681 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251807    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251810    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251813    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251816    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251819    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251822    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251824    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251831    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251834    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251837    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251840    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251843    2581 flags.go:64] FLAG: --enable-server="true"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251846    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251851    2581 flags.go:64] FLAG: --event-burst="100"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251854    2581 flags.go:64] FLAG: --event-qps="50"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251859    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251862    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251865    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251869    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251872    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251875    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251878    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251881    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251883    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251886    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 18:50:31.256220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251889    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251892    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251895    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251898    2581 flags.go:64] FLAG: --feature-gates=""
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251901    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251904    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251907    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251910    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251913    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251916    2581 flags.go:64] FLAG: --help="false"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251919    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251922    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251925    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251928    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251931    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251934    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251937    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251939    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251942    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251945    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251948    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251952    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251956    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251959    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 18:50:31.256928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251962    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251965    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251968    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251971    2581 flags.go:64] FLAG: --lock-file=""
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251973    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251976    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251979    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251984    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251987    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251990    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251992    2581 flags.go:64] FLAG: --logging-format="text"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251995    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.251999    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252002    2581 flags.go:64] FLAG: --manifest-url=""
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252005    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252009    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252012    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252015    2581 flags.go:64] FLAG: --max-pods="110"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252018    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252021    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252024    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252027    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252030    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252033    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252036    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 18:50:31.257494 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252042    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252045    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252048    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252051    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252054    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252060    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252064    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252067    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252070    2581 flags.go:64] FLAG: --port="10250"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252073    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252076    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5036749da2f4c2b"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252080    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252083    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252086    2581 flags.go:64] FLAG: --register-node="true"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252088    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252092    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252094    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252097    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252100    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252104    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252106    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252109    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252112    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252115    2581 flags.go:64] FLAG: --runonce="false"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252118    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 18:50:31.258088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252121    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252124    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252127    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252129    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252132    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252135    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252138    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252141    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252144    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252147    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252149    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252152    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252155    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252159    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252164    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252167    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252170    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252174    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252177    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252180    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252183    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252186    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252189    2581 flags.go:64] FLAG: --v="2"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252193    2581 flags.go:64] FLAG: --version="false"
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252197    2581 flags.go:64] FLAG: --vmodule=""
Jul 13 18:50:31.258930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252200    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.252203    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252345    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252349    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252351    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252355    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252358    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252360    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252363    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252366    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252369    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252371    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252374    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252376    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252379    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252381    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252384    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252388    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252392    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252394    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:50:31.259992 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252399    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252401    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252404    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252406    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252409    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252412    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252416    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252419    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252422    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252424    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252427    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252429    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252432    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252434    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252437    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252440    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252442    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252445    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252447    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252449    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:50:31.260850 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252452    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252455    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252458    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252460    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252463    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252465    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252468    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252470    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252472    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252476    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252479    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252481    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252485    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252487    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252490    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252493    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252495    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252498    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252500    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252503    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:50:31.261730 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252506    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252508    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252511    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252513    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252516    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252518    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252521    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252523    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252526    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252528    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252531    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252533    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252537    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252541    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252543    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252546    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252549    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252566    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252569    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:50:31.262668 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252574    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252577    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252582    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252585    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252587    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252591    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252594    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252596    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252599    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252601    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252604    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252606    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252609    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252612    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252614    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252617    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252620    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252622    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252625    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252627    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:50:31.263169 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252630    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.252632    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.253362    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.261007    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.261025    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.261043    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.261052    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261141    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261149    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261154    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261158    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261163    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261168    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261172    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:50:31.263826 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261177    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261181    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261186    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261190    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261195    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261199    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261203    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261208    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261212    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261216    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261221    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261226    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261231    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261235    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261239    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261243    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261247    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261252    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261256    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261261    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:50:31.264175 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261265    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261269    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261274    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261278    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261283    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261287    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261291    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261296    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261300    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261304    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261309    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261313    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261317    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261322    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261326    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261331    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261336    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261340    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261344    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261349    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:50:31.265004 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261354    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261358    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261362    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261366    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261371    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261375    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261379    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261384    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261388    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261392    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261399    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261405    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261410    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261416    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261421    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261426    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261431    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261435    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261439    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261443    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:50:31.265764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261447    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261451    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261455    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261459    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261464    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261468    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261472    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261476    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261480    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261484    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261491    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261496    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261501    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261505    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261509    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261513    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261518    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261531    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261535    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:50:31.266385 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261539    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261543    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261547    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261568    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261573    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261577    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261581    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261585    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261592    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261596    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261601    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261606    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261610    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.261618    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261829    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:50:31.266905 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261837    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261842    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261846    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261851    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261855    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261859    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261863    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261867    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261872    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261883    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261889    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261893    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261897    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261901    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261905    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261909    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261921    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261925    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261929    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261933    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261937    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:50:31.267387 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261942    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261946    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261950    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261954    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261958    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261962    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261966    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261970    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261974    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261978    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261982    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261986    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261990    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261994    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.261999    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262003    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262007    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262011    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262015    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:50:31.267976 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262021    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262025    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262032    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262039    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262043    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262048    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262052    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262056    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262061    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262075    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262080    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262085    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262089    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262093    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262098    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262102    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262108    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262112    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262117    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:50:31.268485 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262121    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262126    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262131    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262144    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262148    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262167    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262172    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262177    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262181    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262185    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262189    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262195    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262201    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262206    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262210    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262214    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262217    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262224    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262229    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:50:31.269003 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262235    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262240    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262244    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262248    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262260    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262265    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262269    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262273    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262278    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262282    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262286    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262290    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262294    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262298    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262302    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262306    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262311    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262315    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262319    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:50:31.269531 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:31.262323    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:50:31.270317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.262331    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 18:50:31.270317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.262507    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 18:50:31.270317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.266191    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 18:50:31.270317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.267449    2581 server.go:1019] "Starting client certificate rotation"
Jul 13 18:50:31.270317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.267575    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 18:50:31.270317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.267607    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 18:50:31.271917 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.271895    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 18:50:31.273340 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.273321    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 18:50:31.283137 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.283122    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 13 18:50:31.285126 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.285113    2581 log.go:25] "Validated CRI v1 image API"
Jul 13 18:50:31.285870 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.285853    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 18:50:31.288870 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.288847    2581 fs.go:135] Filesystem UUIDs: map[3e5b3040-af16-4904-ae20-5bbff146bd28:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9d21dfeb-dc2f-4067-9158-9f58721a7921:/dev/nvme0n1p4]
Jul 13 18:50:31.288922 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.288870    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 18:50:31.293585 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.293567    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 18:50:31.293931 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.293825    2581 manager.go:216] Machine: {Timestamp:2026-07-13 18:50:31.292575624 +0000 UTC m=+0.339770771 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091159 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20a696f69d723ec51ee70f168655ba SystemUUID:ec20a696-f69d-723e-c51e-e70f168655ba BootID:e5b822e9-32a1-43f5-bfac-cd36a9140fde Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f5:2b:36:da:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f5:2b:36:da:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:23:e1:64:33:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 18:50:31.293931 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.293929    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 18:50:31.294060 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.294048    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 18:50:31.296263 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.296238    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 18:50:31.296395 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.296265    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-48.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 18:50:31.297565 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.297547    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 18:50:31.297603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.297570    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 18:50:31.297603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.297580    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 18:50:31.297603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.297593    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 18:50:31.298768 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.298757    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 18:50:31.301526 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301514    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 18:50:31.301656 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301646    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 18:50:31.301690 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301679    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 18:50:31.301690 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301689    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 18:50:31.301750 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301701    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 18:50:31.301750 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301710    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 13 18:50:31.301750 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.301719    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 18:50:31.302115 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.302104    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 18:50:31.302163 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.302122    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 18:50:31.307178 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.307143    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 18:50:31.308961 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.308942    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 18:50:31.309030 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.308987    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 18:50:31.309153 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309142    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 18:50:31.309207 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309157    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 18:50:31.309207 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309163    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 18:50:31.309207 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309169    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 18:50:31.309207 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309178    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 18:50:31.309207 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309183    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 18:50:31.309866 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309854    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 18:50:31.309909 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309868    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 18:50:31.309909 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309877    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 18:50:31.309909 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309882    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 18:50:31.309909 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309891    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 18:50:31.309909 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.309899    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 18:50:31.310654 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.310641    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 18:50:31.310698 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.310659    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 18:50:31.313957 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.313943    2581 server.go:1268] "Started kubelet"
Jul 13 18:50:31.314060 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.314003    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 18:50:31.314206 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.314047    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 18:50:31.314206 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.314116    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 18:50:31.314813 ip-10-0-134-48 systemd[1]: Started Kubernetes Kubelet.
Jul 13 18:50:31.315363 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.315246    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 18:50:31.316125 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.316106    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-48.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 18:50:31.316210 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.316138    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 18:50:31.316271 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.316136    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 18:50:31.316271 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.316137    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 18:50:31.318871 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.318855    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 18:50:31.319904 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.319887    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 18:50:31.319974 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.319910    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 18:50:31.320029 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.319975    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.320493 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320476    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 18:50:31.320493 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320476    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 18:50:31.320624 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320503    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 18:50:31.320671 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320654    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 18:50:31.320671 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320662    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 18:50:31.320770 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320701    2581 factory.go:55] Registering systemd factory
Jul 13 18:50:31.320770 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320717    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 13 18:50:31.320917 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320902    2581 factory.go:153] Registering CRI-O factory
Jul 13 18:50:31.320917 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320919    2581 factory.go:223] Registration of the crio container factory successfully
Jul 13 18:50:31.321062 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320968    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 18:50:31.321062 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.320993    2581 factory.go:103] Registering Raw factory
Jul 13 18:50:31.321062 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.321007    2581 manager.go:1196] Started watching for new ooms in manager
Jul 13 18:50:31.321372 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.321360    2581 manager.go:349] Starting recovery of all containers
Jul 13 18:50:31.322359 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.322335    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fpc74"
Jul 13 18:50:31.325739 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.325711    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 18:50:31.325873 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.325778    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 18:50:31.328764 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.325503    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-48.ec2.internal.18c1ee04afe7e51b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-48.ec2.internal,UID:ip-10-0-134-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-48.ec2.internal,},FirstTimestamp:2026-07-13 18:50:31.313917211 +0000 UTC m=+0.361112357,LastTimestamp:2026-07-13 18:50:31.313917211 +0000 UTC m=+0.361112357,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-48.ec2.internal,}"
Jul 13 18:50:31.329410 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.329385    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fpc74"
Jul 13 18:50:31.334197 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.334179    2581 manager.go:354] Recovery completed
Jul 13 18:50:31.338497 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.338483    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.340263 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340241    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.340329 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340282    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.340329 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340297    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.340811 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340796    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 18:50:31.340811 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340809    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 18:50:31.340900 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340824    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 18:50:31.340900 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.340868    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.341550 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.341537    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.341638 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.341588    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.341638 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.341600    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.341919 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.341909    2581 policy_none.go:49] "None policy: Start"
Jul 13 18:50:31.341952 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.341924    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 18:50:31.341952 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.341934    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 18:50:31.342672 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.342613    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-48.ec2.internal.18c1ee04b179e4c1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-48.ec2.internal,UID:ip-10-0-134-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-48.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-48.ec2.internal,},FirstTimestamp:2026-07-13 18:50:31.340262593 +0000 UTC m=+0.387457740,LastTimestamp:2026-07-13 18:50:31.340262593 +0000 UTC m=+0.387457740,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-48.ec2.internal,}"
Jul 13 18:50:31.343010 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.342998    2581 policy_none.go:47] "Start"
Jul 13 18:50:31.389478 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.389279    2581 manager.go:337] "Starting Device Plugin manager"
Jul 13 18:50:31.389607 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.389512    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 18:50:31.389607 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.389527    2581 server.go:87] "Starting device plugin registration server"
Jul 13 18:50:31.389786 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.389774    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 18:50:31.389829 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.389788    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 18:50:31.389932 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.389912    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 18:50:31.390024 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.390011    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 18:50:31.390024 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.390021    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 18:50:31.390539 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.390516    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 18:50:31.390653 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.390590    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.490699 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.490653    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.492530 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.492513    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.492748 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.492550    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.492748 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.492582    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.492748 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.492619    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.499423 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.499396    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 18:50:31.500705 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.500683    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 18:50:31.500794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.500718    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 18:50:31.500794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.500740    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 18:50:31.500794 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.500775    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 18:50:31.502350 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.502328    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.502428 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.502358    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-48.ec2.internal\": node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.506937 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.506921    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:50:31.542039 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.542019    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.601860 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.601822    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"]
Jul 13 18:50:31.601935 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.601914    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.602982 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.602964    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.603062 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.603002    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.603062 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.603018    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.604273 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.604257    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.604385 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.604370    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.604437 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.604400    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.607679 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.607656    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.607763 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.607690    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.607763 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.607702    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.607763 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.607656    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.607860 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.607769    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.607860 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.607783    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.609193 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.609180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.609237 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.609205    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:50:31.609906 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.609890    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:50:31.609984 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.609920    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:50:31.609984 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.609933    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:50:31.622792 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.622773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/242362b85a52f1c40232c006779f8a86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal\" (UID: \"242362b85a52f1c40232c006779f8a86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.622869 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.622798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/242362b85a52f1c40232c006779f8a86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal\" (UID: \"242362b85a52f1c40232c006779f8a86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.622869 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.622817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/648a1691dcc256654d0f1487adfe8cf6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-48.ec2.internal\" (UID: \"648a1691dcc256654d0f1487adfe8cf6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.634321 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.634301    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-48.ec2.internal\" not found" node="ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.638949 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.638933    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-48.ec2.internal\" not found" node="ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.643061 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.643045    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.723980 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.723957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/242362b85a52f1c40232c006779f8a86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal\" (UID: \"242362b85a52f1c40232c006779f8a86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.724072 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.723984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/648a1691dcc256654d0f1487adfe8cf6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-48.ec2.internal\" (UID: \"648a1691dcc256654d0f1487adfe8cf6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.724072 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.724003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/242362b85a52f1c40232c006779f8a86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal\" (UID: \"242362b85a52f1c40232c006779f8a86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.724072 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.724030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/242362b85a52f1c40232c006779f8a86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal\" (UID: \"242362b85a52f1c40232c006779f8a86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.724072 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.724056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/242362b85a52f1c40232c006779f8a86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal\" (UID: \"242362b85a52f1c40232c006779f8a86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.724220 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.724081    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/648a1691dcc256654d0f1487adfe8cf6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-48.ec2.internal\" (UID: \"648a1691dcc256654d0f1487adfe8cf6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.743214 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.743161    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.844087 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.844043    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:31.936184 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.936150    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.941469 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:31.941436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:31.944261 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:31.944247    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.045111 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.045014    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.145545 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.145518    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.145697 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.145680    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:50:32.246275 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.246238    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.268017 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.267992    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 18:50:32.268156 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.268136    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 18:50:32.268197 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.268180    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 18:50:32.333109 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.333027    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 18:45:32 +0000 UTC" deadline="2028-04-29 07:39:39.081006242 +0000 UTC"
Jul 13 18:50:32.333109 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.333068    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15732h49m6.747941927s"
Jul 13 18:50:32.346539 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.346518    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.446985 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.446951    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.547893 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.547856    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.613522 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:32.613491    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod242362b85a52f1c40232c006779f8a86.slice/crio-06eb1027f02fec9975cbcc165c3cc6fee88c0fd947e85084c1e1a0ab8044c11e WatchSource:0}: Error finding container 06eb1027f02fec9975cbcc165c3cc6fee88c0fd947e85084c1e1a0ab8044c11e: Status 404 returned error can't find the container with id 06eb1027f02fec9975cbcc165c3cc6fee88c0fd947e85084c1e1a0ab8044c11e
Jul 13 18:50:32.617842 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.617825    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 18:50:32.649547 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:32.649519    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-48.ec2.internal\" not found"
Jul 13 18:50:32.746429 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.746403    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:50:32.821008 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.820977    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:32.831945 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.831919    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:50:32.850439 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.850274    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 18:50:32.851797 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.851438    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal"
Jul 13 18:50:32.860361 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:32.860303    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 18:50:32.954530 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:32.954500    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648a1691dcc256654d0f1487adfe8cf6.slice/crio-cbd7a886b0e44d12f48aad0bf69e9b16deeaf64cc9a12fc43ffb8779babbaab3 WatchSource:0}: Error finding container cbd7a886b0e44d12f48aad0bf69e9b16deeaf64cc9a12fc43ffb8779babbaab3: Status 404 returned error can't find the container with id cbd7a886b0e44d12f48aad0bf69e9b16deeaf64cc9a12fc43ffb8779babbaab3
Jul 13 18:50:33.304298 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.304213    2581 apiserver.go:52] "Watching apiserver"
Jul 13 18:50:33.309585 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.309552    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 18:50:33.310006 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.309982    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-6bwhr","kube-system/konnectivity-agent-t4pf9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86","openshift-cluster-node-tuning-operator/tuned-4dq2k","openshift-image-registry/node-ca-97w5n","openshift-multus/multus-additional-cni-plugins-zcdgx","openshift-ovn-kubernetes/ovnkube-node-frplb","kube-system/global-pull-secret-syncer-lvmn2","kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal","openshift-dns/node-resolver-r4459","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal","openshift-multus/multus-prhls","openshift-multus/network-metrics-daemon-5xfpm","openshift-network-diagnostics/network-check-target-77cd8"]
Jul 13 18:50:33.311577 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.311547    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.312921 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.312900    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.313036 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.312980    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:33.313926 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.313906    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 18:50:33.314307 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314217    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.314307 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314259    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 18:50:33.314454 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314309    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 18:50:33.314454 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314323    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 18:50:33.314454 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.314649 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314559    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.314803 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.314789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ccpqr\""
Jul 13 18:50:33.315553 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.315536    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.316099 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.316081    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 18:50:33.316406 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.316367    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-67rdr\""
Jul 13 18:50:33.317015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.316993    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.317402 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.317240    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.317509 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.317480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.318029 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.317984    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.318029 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.318011    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pbm2z\""
Jul 13 18:50:33.318199 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.318049    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.318913 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.318895    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.319603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.319327    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 18:50:33.319725 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.319667    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.319725 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.319683    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pjpbv\""
Jul 13 18:50:33.319869 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.319846    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.320294 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.320273    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 18:50:33.320874 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.320630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:33.320874 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.320739    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:33.321044 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.321010    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 18:50:33.321044 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.321023    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-95n8t\""
Jul 13 18:50:33.321260 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.321222    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 18:50:33.321454 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.321431    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.321813 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.321795    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.322226 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.321972    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 18:50:33.322226 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.322167    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.324937 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.324903    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.325251 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.325228    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.325338 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.325303    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xlsgp\""
Jul 13 18:50:33.325589 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.325558    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.325693 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.325237    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 18:50:33.327208 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.327118    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.327208 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.327178    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 18:50:33.327364 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.327323    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 18:50:33.327629 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.327604    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zjmf9\""
Jul 13 18:50:33.328818 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.328795    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.328929 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.328870    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:33.329099 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.329080    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 18:50:33.329231 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.329214    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 18:50:33.329327 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.329311    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6dgls\""
Jul 13 18:50:33.330553 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.330535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.332474 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.332456    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 18:50:33.332563 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.332547    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2cgbp\""
Jul 13 18:50:33.332791 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.332771    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 18:50:33.333522 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.333620 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-env-overrides\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.333620 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-kubernetes\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.333620 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e291613f-af89-4da4-827c-aaf0c2819787-serviceca\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.333762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d71dc73c-f80c-4746-887c-4b0d18073d53-iptables-alerter-script\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.333762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-os-release\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.333762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.333762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-slash\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.333762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lkgc\" (UniqueName: \"kubernetes.io/projected/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-kube-api-access-9lkgc\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-872xt\" (UniqueName: \"kubernetes.io/projected/da08ff36-fa0f-4526-a23e-e2c759749ce6-kube-api-access-872xt\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvjzf\" (UniqueName: \"kubernetes.io/projected/648f005d-b22e-4abd-a731-89b710f8d991-kube-api-access-wvjzf\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-modprobe-d\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-host\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbwrh\" (UniqueName: \"kubernetes.io/projected/2227d0e7-3b39-46be-8b94-fdc23bd64eac-kube-api-access-kbwrh\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.333961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d71dc73c-f80c-4746-887c-4b0d18073d53-host-slash\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-run-netns\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/648f005d-b22e-4abd-a731-89b710f8d991-ovn-node-metrics-cert\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-var-lib-kubelet\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysctl-d\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-node-log\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2227d0e7-3b39-46be-8b94-fdc23bd64eac-tmp\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-ovnkube-script-lib\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-tuned\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e291613f-af89-4da4-827c-aaf0c2819787-host\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-registration-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-run-ovn-kubernetes\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-lib-modules\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-cni-bin\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-cni-netd\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-kubelet\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysconfig\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-run\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c5rb\" (UniqueName: \"kubernetes.io/projected/d71dc73c-f80c-4746-887c-4b0d18073d53-kube-api-access-6c5rb\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-systemd-units\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-systemd\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-var-lib-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-etc-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-sys\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.334996 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-sys-fs\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-systemd\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334899    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-ovn\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-ovnkube-config\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-system-cni-dir\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cnibin\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.334997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-log-socket\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.335024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-socket-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.335049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-device-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.335073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2nlv\" (UniqueName: \"kubernetes.io/projected/b09f0a67-27f1-4660-a46a-c9d30c5952ed-kube-api-access-f2nlv\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.335096    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysctl-conf\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.335752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.335115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7gj8\" (UniqueName: \"kubernetes.io/projected/e291613f-af89-4da4-827c-aaf0c2819787-kube-api-access-h7gj8\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.358411 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.358389    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t7hlf"
Jul 13 18:50:33.368066 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.368045    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t7hlf"
Jul 13 18:50:33.421505 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.421470    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 18:50:33.435538 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-socket-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.435708 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-device-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.435708 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2nlv\" (UniqueName: \"kubernetes.io/projected/b09f0a67-27f1-4660-a46a-c9d30c5952ed-kube-api-access-f2nlv\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.435708 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-k8s-cni-cncf-io\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.435708 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysctl-conf\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.435708 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-device-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.435708 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7gj8\" (UniqueName: \"kubernetes.io/projected/e291613f-af89-4da4-827c-aaf0c2819787-kube-api-access-h7gj8\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.436015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-socket-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.436015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysctl-conf\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-cni-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-multus-certs\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ltq\" (UniqueName: \"kubernetes.io/projected/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-kube-api-access-c8ltq\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.435962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-env-overrides\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-kubernetes\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e291613f-af89-4da4-827c-aaf0c2819787-serviceca\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb3ba19f-0cbf-4239-b5f2-7cb95490561a-konnectivity-ca\") pod \"konnectivity-agent-t4pf9\" (UID: \"cb3ba19f-0cbf-4239-b5f2-7cb95490561a\") " pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-cni-bin\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-kubernetes\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d71dc73c-f80c-4746-887c-4b0d18073d53-iptables-alerter-script\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-os-release\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.436271 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-slash\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-tmp-dir\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-etc-kubernetes\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27995e44-0dd7-4291-91d5-d0823bd49fb5-kubelet-config\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-cni-multus\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lkgc\" (UniqueName: \"kubernetes.io/projected/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-kube-api-access-9lkgc\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-872xt\" (UniqueName: \"kubernetes.io/projected/da08ff36-fa0f-4526-a23e-e2c759749ce6-kube-api-access-872xt\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436513    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-env-overrides\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-kubelet\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-daemon-config\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvjzf\" (UniqueName: \"kubernetes.io/projected/648f005d-b22e-4abd-a731-89b710f8d991-kube-api-access-wvjzf\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-modprobe-d\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-host\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e291613f-af89-4da4-827c-aaf0c2819787-serviceca\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwrh\" (UniqueName: \"kubernetes.io/projected/2227d0e7-3b39-46be-8b94-fdc23bd64eac-kube-api-access-kbwrh\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d71dc73c-f80c-4746-887c-4b0d18073d53-host-slash\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-run-netns\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/648f005d-b22e-4abd-a731-89b710f8d991-ovn-node-metrics-cert\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436884    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-host\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-var-lib-kubelet\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysctl-d\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-var-lib-kubelet\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.436988 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.436922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-modprobe-d\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysctl-d\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d71dc73c-f80c-4746-887c-4b0d18073d53-iptables-alerter-script\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437111    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-run-netns\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-cnibin\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-node-log\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437164    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-slash\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d71dc73c-f80c-4746-887c-4b0d18073d53-host-slash\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2227d0e7-3b39-46be-8b94-fdc23bd64eac-tmp\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-os-release\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-node-log\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27995e44-0dd7-4291-91d5-d0823bd49fb5-dbus\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.437794 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-conf-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-ovnkube-script-lib\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-tuned\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437556    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e291613f-af89-4da4-827c-aaf0c2819787-host\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-registration-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-cni-binary-copy\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e291613f-af89-4da4-827c-aaf0c2819787-host\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437729    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-registration-dir\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-socket-dir-parent\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-run-ovn-kubernetes\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-lib-modules\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-run-ovn-kubernetes\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-cni-bin\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-cni-netd\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.438558 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.437982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-kubelet\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysconfig\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-run\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c5rb\" (UniqueName: \"kubernetes.io/projected/d71dc73c-f80c-4746-887c-4b0d18073d53-kube-api-access-6c5rb\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-systemd-units\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-systemd\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb3ba19f-0cbf-4239-b5f2-7cb95490561a-agent-certs\") pod \"konnectivity-agent-t4pf9\" (UID: \"cb3ba19f-0cbf-4239-b5f2-7cb95490561a\") " pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-hosts-file\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-system-cni-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438268    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-os-release\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-hostroot\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-lib-modules\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-var-lib-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-kubelet\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.439383 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-var-lib-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-etc-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-sys\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-sys-fs\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-cni-bin\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-run\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-netns\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-systemd\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-host-cni-netd\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-systemd-units\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-ovn\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-etc-openvswitch\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvb4f\" (UniqueName: \"kubernetes.io/projected/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-kube-api-access-gvb4f\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-ovnkube-script-lib\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-ovn\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-ovnkube-config\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.440324 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.438652    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-sys\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-sysconfig\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b09f0a67-27f1-4660-a46a-c9d30c5952ed-sys-fs\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.438811    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:33.938777709 +0000 UTC m=+2.985972861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-run-systemd\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-system-cni-dir\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-systemd\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cnibin\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.438988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-log-socket\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.439071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-system-cni-dir\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.439087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/648f005d-b22e-4abd-a731-89b710f8d991-log-socket\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.439140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-cnibin\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.441230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.439944    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/648f005d-b22e-4abd-a731-89b710f8d991-ovnkube-config\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.441830 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.441757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2227d0e7-3b39-46be-8b94-fdc23bd64eac-etc-tuned\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.441896 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.441877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/648f005d-b22e-4abd-a731-89b710f8d991-ovn-node-metrics-cert\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.442320 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.442288    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2227d0e7-3b39-46be-8b94-fdc23bd64eac-tmp\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.457937 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.457909    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:50:33.457937 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.457936    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:50:33.457937 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.457947    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:33.458242 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.458011    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:50:33.95799118 +0000 UTC m=+3.005186315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:33.458242 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.458096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c5rb\" (UniqueName: \"kubernetes.io/projected/d71dc73c-f80c-4746-887c-4b0d18073d53-kube-api-access-6c5rb\") pod \"iptables-alerter-6bwhr\" (UID: \"d71dc73c-f80c-4746-887c-4b0d18073d53\") " pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.459948 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.459444    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2nlv\" (UniqueName: \"kubernetes.io/projected/b09f0a67-27f1-4660-a46a-c9d30c5952ed-kube-api-access-f2nlv\") pod \"aws-ebs-csi-driver-node-v8g86\" (UID: \"b09f0a67-27f1-4660-a46a-c9d30c5952ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.460729 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.460684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lkgc\" (UniqueName: \"kubernetes.io/projected/38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e-kube-api-access-9lkgc\") pod \"multus-additional-cni-plugins-zcdgx\" (UID: \"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e\") " pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.460950 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.460924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbwrh\" (UniqueName: \"kubernetes.io/projected/2227d0e7-3b39-46be-8b94-fdc23bd64eac-kube-api-access-kbwrh\") pod \"tuned-4dq2k\" (UID: \"2227d0e7-3b39-46be-8b94-fdc23bd64eac\") " pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.462955 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.462932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7gj8\" (UniqueName: \"kubernetes.io/projected/e291613f-af89-4da4-827c-aaf0c2819787-kube-api-access-h7gj8\") pod \"node-ca-97w5n\" (UID: \"e291613f-af89-4da4-827c-aaf0c2819787\") " pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.463102 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.463078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvjzf\" (UniqueName: \"kubernetes.io/projected/648f005d-b22e-4abd-a731-89b710f8d991-kube-api-access-wvjzf\") pod \"ovnkube-node-frplb\" (UID: \"648f005d-b22e-4abd-a731-89b710f8d991\") " pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.465793 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.465770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-872xt\" (UniqueName: \"kubernetes.io/projected/da08ff36-fa0f-4526-a23e-e2c759749ce6-kube-api-access-872xt\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.508457 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.508393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal" event={"ID":"242362b85a52f1c40232c006779f8a86","Type":"ContainerStarted","Data":"06eb1027f02fec9975cbcc165c3cc6fee88c0fd947e85084c1e1a0ab8044c11e"}
Jul 13 18:50:33.509587 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.509541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal" event={"ID":"648a1691dcc256654d0f1487adfe8cf6","Type":"ContainerStarted","Data":"cbd7a886b0e44d12f48aad0bf69e9b16deeaf64cc9a12fc43ffb8779babbaab3"}
Jul 13 18:50:33.539792 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.539760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-cni-binary-copy\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.539963 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.539803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-socket-dir-parent\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.539963 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.539927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-socket-dir-parent\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540083 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.539966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb3ba19f-0cbf-4239-b5f2-7cb95490561a-agent-certs\") pod \"konnectivity-agent-t4pf9\" (UID: \"cb3ba19f-0cbf-4239-b5f2-7cb95490561a\") " pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.540083 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.539994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-hosts-file\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.540083 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.540083 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-system-cni-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540083 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-os-release\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-hostroot\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-netns\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvb4f\" (UniqueName: \"kubernetes.io/projected/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-kube-api-access-gvb4f\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.540217    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-k8s-cni-cncf-io\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-cni-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.540288    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:34.040267209 +0000 UTC m=+3.087462364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:33.540332 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-multus-certs\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ltq\" (UniqueName: \"kubernetes.io/projected/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-kube-api-access-c8ltq\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb3ba19f-0cbf-4239-b5f2-7cb95490561a-konnectivity-ca\") pod \"konnectivity-agent-t4pf9\" (UID: \"cb3ba19f-0cbf-4239-b5f2-7cb95490561a\") " pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-cni-bin\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-tmp-dir\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-cni-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-etc-kubernetes\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27995e44-0dd7-4291-91d5-d0823bd49fb5-kubelet-config\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-cni-multus\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-os-release\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540539    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-kubelet\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-cni-binary-copy\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540561    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-hostroot\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-daemon-config\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-cnibin\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-k8s-cni-cncf-io\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-netns\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27995e44-0dd7-4291-91d5-d0823bd49fb5-dbus\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.540779 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540682    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-etc-kubernetes\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-conf-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-system-cni-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540811    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-run-multus-certs\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-conf-dir\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27995e44-0dd7-4291-91d5-d0823bd49fb5-kubelet-config\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27995e44-0dd7-4291-91d5-d0823bd49fb5-dbus\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-cni-multus\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.540980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-tmp-dir\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.541085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-cnibin\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.541135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-hosts-file\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.541318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-cni-bin\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.541365    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-host-var-lib-kubelet\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.541426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-multus-daemon-config\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.541790 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.541680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb3ba19f-0cbf-4239-b5f2-7cb95490561a-konnectivity-ca\") pod \"konnectivity-agent-t4pf9\" (UID: \"cb3ba19f-0cbf-4239-b5f2-7cb95490561a\") " pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.543517 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.543492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb3ba19f-0cbf-4239-b5f2-7cb95490561a-agent-certs\") pod \"konnectivity-agent-t4pf9\" (UID: \"cb3ba19f-0cbf-4239-b5f2-7cb95490561a\") " pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.553440 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.553394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ltq\" (UniqueName: \"kubernetes.io/projected/a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1-kube-api-access-c8ltq\") pod \"multus-prhls\" (UID: \"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1\") " pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.553632 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.553608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvb4f\" (UniqueName: \"kubernetes.io/projected/d4f1d1b0-33f6-4248-b57e-e9af5bd4af07-kube-api-access-gvb4f\") pod \"node-resolver-r4459\" (UID: \"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07\") " pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.624615 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.624515    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:33.634839 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.634812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86"
Jul 13 18:50:33.642869 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.642845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4dq2k"
Jul 13 18:50:33.648741 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.648714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97w5n"
Jul 13 18:50:33.656545 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.656525    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zcdgx"
Jul 13 18:50:33.664480 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.664461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bwhr"
Jul 13 18:50:33.670728 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.670709    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:33.678564 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.678546    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r4459"
Jul 13 18:50:33.683503 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.683483    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-prhls"
Jul 13 18:50:33.813799 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.813770    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09f0a67_27f1_4660_a46a_c9d30c5952ed.slice/crio-d4d94dd84eeecd45a91747181507258acdcbb50c42bd98b16df2b57b0ce4db90 WatchSource:0}: Error finding container d4d94dd84eeecd45a91747181507258acdcbb50c42bd98b16df2b57b0ce4db90: Status 404 returned error can't find the container with id d4d94dd84eeecd45a91747181507258acdcbb50c42bd98b16df2b57b0ce4db90
Jul 13 18:50:33.815342 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.815218    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71dc73c_f80c_4746_887c_4b0d18073d53.slice/crio-be308d37b544fffa25670ab91886c07013cfedb6bdfdc628a44c808cd620060f WatchSource:0}: Error finding container be308d37b544fffa25670ab91886c07013cfedb6bdfdc628a44c808cd620060f: Status 404 returned error can't find the container with id be308d37b544fffa25670ab91886c07013cfedb6bdfdc628a44c808cd620060f
Jul 13 18:50:33.818767 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.818565    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f1d1b0_33f6_4248_b57e_e9af5bd4af07.slice/crio-991296084d6e667a07c89382c77c905396b23ad6ae93a88cfd051d497abc7ced WatchSource:0}: Error finding container 991296084d6e667a07c89382c77c905396b23ad6ae93a88cfd051d497abc7ced: Status 404 returned error can't find the container with id 991296084d6e667a07c89382c77c905396b23ad6ae93a88cfd051d497abc7ced
Jul 13 18:50:33.819042 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.818910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ba7735_1318_4b8f_955e_4f8eeb6eb9f1.slice/crio-c250804aeb4b1c2e286bf5b565145c098ff97946e5aa45329418f696d0af3c9c WatchSource:0}: Error finding container c250804aeb4b1c2e286bf5b565145c098ff97946e5aa45329418f696d0af3c9c: Status 404 returned error can't find the container with id c250804aeb4b1c2e286bf5b565145c098ff97946e5aa45329418f696d0af3c9c
Jul 13 18:50:33.819683 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.819659    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648f005d_b22e_4abd_a731_89b710f8d991.slice/crio-b76362d22801b119daf18f2d62142cd6606336aab058febf11e878f383dfde3d WatchSource:0}: Error finding container b76362d22801b119daf18f2d62142cd6606336aab058febf11e878f383dfde3d: Status 404 returned error can't find the container with id b76362d22801b119daf18f2d62142cd6606336aab058febf11e878f383dfde3d
Jul 13 18:50:33.821039 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.821010    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode291613f_af89_4da4_827c_aaf0c2819787.slice/crio-48fea95b0efed5920695272da94970a8e8e11767c20707970a8db3bf247ddf19 WatchSource:0}: Error finding container 48fea95b0efed5920695272da94970a8e8e11767c20707970a8db3bf247ddf19: Status 404 returned error can't find the container with id 48fea95b0efed5920695272da94970a8e8e11767c20707970a8db3bf247ddf19
Jul 13 18:50:33.822155 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.822122    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2227d0e7_3b39_46be_8b94_fdc23bd64eac.slice/crio-69d18d2c259f87735aa427d69ac708fe49c2addf74c6b84c36f5492edbc08a19 WatchSource:0}: Error finding container 69d18d2c259f87735aa427d69ac708fe49c2addf74c6b84c36f5492edbc08a19: Status 404 returned error can't find the container with id 69d18d2c259f87735aa427d69ac708fe49c2addf74c6b84c36f5492edbc08a19
Jul 13 18:50:33.823611 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:50:33.823588    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3ba19f_0cbf_4239_b5f2_7cb95490561a.slice/crio-947516183ba2dfd858d13fe08ec37d8e8d6caf08f1998c1a59736f88acf3ab2b WatchSource:0}: Error finding container 947516183ba2dfd858d13fe08ec37d8e8d6caf08f1998c1a59736f88acf3ab2b: Status 404 returned error can't find the container with id 947516183ba2dfd858d13fe08ec37d8e8d6caf08f1998c1a59736f88acf3ab2b
Jul 13 18:50:33.831791 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.831773    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:50:33.944230 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:33.944133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:33.944388 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.944280    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:33.944388 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:33.944345    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:34.944325129 +0000 UTC m=+3.991520285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:34.045446 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.045414    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:34.045629 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.045472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:34.045709 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.045671    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:34.045762 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.045725    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:50:34.045762 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.045744    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:35.04572011 +0000 UTC m=+4.092915258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:34.045762 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.045756    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:50:34.045939 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.045772    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:34.045939 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.045826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:50:35.045808585 +0000 UTC m=+4.093003724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:34.371060 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.370733    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:45:34 +0000 UTC" deadline="2028-03-20 06:38:55.006107522 +0000 UTC"
Jul 13 18:50:34.371060 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.371015    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14771h48m20.635100369s"
Jul 13 18:50:34.501644 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.501447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:34.502725 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.501934    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:34.502725 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.502387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:34.502725 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.502509    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:34.502725 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.502570    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:34.502725 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.502664    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:34.516087 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.515961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"b76362d22801b119daf18f2d62142cd6606336aab058febf11e878f383dfde3d"}
Jul 13 18:50:34.517567 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.517536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-prhls" event={"ID":"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1","Type":"ContainerStarted","Data":"c250804aeb4b1c2e286bf5b565145c098ff97946e5aa45329418f696d0af3c9c"}
Jul 13 18:50:34.520386 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.520312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86" event={"ID":"b09f0a67-27f1-4660-a46a-c9d30c5952ed","Type":"ContainerStarted","Data":"d4d94dd84eeecd45a91747181507258acdcbb50c42bd98b16df2b57b0ce4db90"}
Jul 13 18:50:34.522356 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.522275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerStarted","Data":"9a3440ecffa3513e0f07bdbd93996f89f3c77f2eba08b2ae36a6523d17908f1c"}
Jul 13 18:50:34.524095 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.524020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t4pf9" event={"ID":"cb3ba19f-0cbf-4239-b5f2-7cb95490561a","Type":"ContainerStarted","Data":"947516183ba2dfd858d13fe08ec37d8e8d6caf08f1998c1a59736f88acf3ab2b"}
Jul 13 18:50:34.527831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.527759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4dq2k" event={"ID":"2227d0e7-3b39-46be-8b94-fdc23bd64eac","Type":"ContainerStarted","Data":"69d18d2c259f87735aa427d69ac708fe49c2addf74c6b84c36f5492edbc08a19"}
Jul 13 18:50:34.536135 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.536108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97w5n" event={"ID":"e291613f-af89-4da4-827c-aaf0c2819787","Type":"ContainerStarted","Data":"48fea95b0efed5920695272da94970a8e8e11767c20707970a8db3bf247ddf19"}
Jul 13 18:50:34.541960 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.541934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r4459" event={"ID":"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07","Type":"ContainerStarted","Data":"991296084d6e667a07c89382c77c905396b23ad6ae93a88cfd051d497abc7ced"}
Jul 13 18:50:34.544881 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.544855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bwhr" event={"ID":"d71dc73c-f80c-4746-887c-4b0d18073d53","Type":"ContainerStarted","Data":"be308d37b544fffa25670ab91886c07013cfedb6bdfdc628a44c808cd620060f"}
Jul 13 18:50:34.551490 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.551461    2581 generic.go:356] "Generic (PLEG): container finished" podID="242362b85a52f1c40232c006779f8a86" containerID="88395f9d82662ab5f61f57341e69d3c757edfe5b49d0ee509eb97d931eaa4156" exitCode=0
Jul 13 18:50:34.551599 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.551500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal" event={"ID":"242362b85a52f1c40232c006779f8a86","Type":"ContainerDied","Data":"88395f9d82662ab5f61f57341e69d3c757edfe5b49d0ee509eb97d931eaa4156"}
Jul 13 18:50:34.906292 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.906261    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:50:34.952625 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:34.952588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:34.952893 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.952849    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:34.952961 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:34.952950    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:36.952929024 +0000 UTC m=+6.000124175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:35.053324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:35.053383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:35.053546    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:35.053631    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:37.053592417 +0000 UTC m=+6.100787558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:35.054043    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:35.054094    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:35.054109    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:35.054196 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:35.054157    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:50:37.054141556 +0000 UTC m=+6.101336714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:35.372178 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:35.371998    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:45:34 +0000 UTC" deadline="2028-04-30 10:08:03.883233768 +0000 UTC"
Jul 13 18:50:35.372178 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:35.372063    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15759h17m28.511175623s"
Jul 13 18:50:36.502322 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:36.501481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:36.502322 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:36.501653    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:36.502322 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:36.502086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:36.502322 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:36.502205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:36.502322 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:36.502250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:36.502322 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:36.502317    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:36.967323 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:36.967232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:36.967498 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:36.967418    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:36.967498 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:36.967483    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:40.96746179 +0000 UTC m=+10.014656940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:37.068446 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:37.068410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:37.068657 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:37.068468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:37.068657 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:37.068635    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:37.068806 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:37.068699    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:41.068680409 +0000 UTC m=+10.115875551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:37.068806 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:37.068792    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:50:37.068806 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:37.068805    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:50:37.068974 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:37.068818    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:37.068974 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:37.068854    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:50:41.068842275 +0000 UTC m=+10.116037412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:38.501858 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:38.501825    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:38.502342 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:38.501974    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:38.502421 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:38.502362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:38.502500 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:38.502467    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:38.502559 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:38.502518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:38.502616 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:38.502597    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:40.501398 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:40.501353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:40.501886 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:40.501358    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:40.501886 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:40.501488    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:40.501886 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:40.501544    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:40.501886 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:40.501374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:40.501886 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:40.501642    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:41.004064 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:41.003989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:41.004278 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.004141    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:41.004278 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.004217    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:49.00419561 +0000 UTC m=+18.051390749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:41.104975 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:41.104931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:41.105177 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:41.104993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:41.105244 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.105196    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:41.105244 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.105210    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:50:41.105244 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.105240    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:50:41.105394 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.105267    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:50:49.10524694 +0000 UTC m=+18.152442081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:41.105394 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.105268    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:41.105394 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:41.105330    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:50:49.105311001 +0000 UTC m=+18.152506143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:42.501885 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:42.501838    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:42.501885 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:42.501869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:42.502437 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:42.501998    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:42.502437 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:42.502090    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:42.502437 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:42.502145    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:42.502437 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:42.502223    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:44.501694 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:44.501602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:44.502092 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:44.501602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:44.502092 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:44.501747    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:44.502092 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:44.501789    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:44.502092 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:44.501602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:44.502092 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:44.501901    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:46.501154 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:46.501118    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:46.501154 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:46.501150    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:46.501670 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:46.501118    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:46.501670 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:46.501268    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:46.501670 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:46.501407    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:46.501670 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:46.501507    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:48.501590 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:48.501539    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:48.501590 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:48.501581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:48.502156 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:48.501539    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:48.502156 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:48.501655    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:48.502156 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:48.501751    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:48.502156 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:48.501838    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:49.067070 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:49.067034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:49.067207 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.067144    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:49.067207 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.067193    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:51:05.067179662 +0000 UTC m=+34.114374795 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:50:49.168197 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:49.168152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:49.168375 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:49.168205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:49.168375 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.168338    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:50:49.168375 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.168373    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:50:49.168515 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.168384    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:49.168515 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.168434    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:51:05.168415188 +0000 UTC m=+34.215610321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:50:49.168515 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.168340    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:49.168515 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:49.168493    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:51:05.16847736 +0000 UTC m=+34.215672494 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:50:50.501769 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:50.501730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:50.502247 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:50.501783    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:50.502247 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:50.501793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:50.502247 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:50.501866    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:50.502247 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:50.501983    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:50.502247 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:50.502071    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:52.502094 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:52.502052    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:52.502548 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:52.502061    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:52.502548 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:52.502192    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:52.502548 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:52.502275    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:52.502548 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:52.502062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:52.502548 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:52.502398    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:54.501772 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.501455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:54.502449 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.501522    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:54.502449 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:54.501848    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:54.502449 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.501529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:54.502449 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:54.501941    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:54.502449 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:54.502015    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:54.595015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.594976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-prhls" event={"ID":"a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1","Type":"ContainerStarted","Data":"30559be453ec2d61bf5ccb86ba6c102dc04a0a36f376c5d3c6e0c0bf67d67caa"}
Jul 13 18:50:54.596364 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.596338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86" event={"ID":"b09f0a67-27f1-4660-a46a-c9d30c5952ed","Type":"ContainerStarted","Data":"8c363dfe98eb3528188d8c6fdbe7d4d7a8e4718e5ad2806a9b2c24c8097132a9"}
Jul 13 18:50:54.597828 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.597804    2581 generic.go:356] "Generic (PLEG): container finished" podID="38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e" containerID="1987ba37c0aa26b8c571057634a9454a9c6726565ab0cc05e038c152429842d4" exitCode=0
Jul 13 18:50:54.597919 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.597870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerDied","Data":"1987ba37c0aa26b8c571057634a9454a9c6726565ab0cc05e038c152429842d4"}
Jul 13 18:50:54.599316 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.599257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t4pf9" event={"ID":"cb3ba19f-0cbf-4239-b5f2-7cb95490561a","Type":"ContainerStarted","Data":"722e2b591c9e8a61539a13301515607ec4c6ccf04fee8b048aa3bdd36a803adc"}
Jul 13 18:50:54.600603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.600582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4dq2k" event={"ID":"2227d0e7-3b39-46be-8b94-fdc23bd64eac","Type":"ContainerStarted","Data":"35deeb08250cc50f93ac0b2ee06426e2e16601baa00d08bd0cd10f97bfed7aa4"}
Jul 13 18:50:54.602042 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.602022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97w5n" event={"ID":"e291613f-af89-4da4-827c-aaf0c2819787","Type":"ContainerStarted","Data":"a32ae70d80c0b33d900dfe483cb2d9055ffea708c73ed942b4e8ce66f9a252a0"}
Jul 13 18:50:54.603349 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.603331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r4459" event={"ID":"d4f1d1b0-33f6-4248-b57e-e9af5bd4af07","Type":"ContainerStarted","Data":"0aee047141944de101ee6a3b90d16f2003df37703ef9df6040f7c2b362b45633"}
Jul 13 18:50:54.604740 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.604719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal" event={"ID":"648a1691dcc256654d0f1487adfe8cf6","Type":"ContainerStarted","Data":"15c3735404f7708934a77bde7a5221b5b0c7e3d5e78c6b42e2b5397d3d2adcd0"}
Jul 13 18:50:54.606468 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.606451    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal" event={"ID":"242362b85a52f1c40232c006779f8a86","Type":"ContainerStarted","Data":"0bb1a92bda5340904f47230f6eff66d97fa410d9986738d29411c62f0382abe1"}
Jul 13 18:50:54.609195 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609179    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 18:50:54.609644 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609621    2581 generic.go:356] "Generic (PLEG): container finished" podID="648f005d-b22e-4abd-a731-89b710f8d991" containerID="d1b0a888b588bbd417ad22a7b7e86e1f251d65fa7ccb5c21d6eb49efb8ed3929" exitCode=1
Jul 13 18:50:54.609743 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"b27f63ccfd4866d5bd16dcec4550a03e3f4a47b80ba00d89eebf79f22dfdadc1"}
Jul 13 18:50:54.609743 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609718    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"73236f24d6dc06f76356041b3d4191aed9315f8e6b1c04d0b2181975b68efc7e"}
Jul 13 18:50:54.609848 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"5322a42d3f9276c35adcc21b0c950e25c8d8e5c21ede9beaec027c24caed2a2e"}
Jul 13 18:50:54.609848 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"17764d2d9b39ad8bd2648b51718e5904dcff62b1c3f23d235b82da0e584ec597"}
Jul 13 18:50:54.609953 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerDied","Data":"d1b0a888b588bbd417ad22a7b7e86e1f251d65fa7ccb5c21d6eb49efb8ed3929"}
Jul 13 18:50:54.609953 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.609922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"b3f5ec6fd45be961153b97909dc6e270e20cae8199bb2e840712750803293cbc"}
Jul 13 18:50:54.679058 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.678940    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-prhls" podStartSLOduration=2.590922609 podStartE2EDuration="22.678920364s" totalImagesPullingTime="20.087997755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.653825896 +0000 UTC m=+23.701021051" watchObservedRunningTime="2026-07-13 18:50:54.678920364 +0000 UTC m=+23.726115521"
Jul 13 18:50:54.709026 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.708969    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-48.ec2.internal" podStartSLOduration=21.708948174 podStartE2EDuration="21.708948174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.679316522 +0000 UTC m=+23.726511680" watchObservedRunningTime="2026-07-13 18:50:54.708948174 +0000 UTC m=+23.756143322"
Jul 13 18:50:54.729760 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.729714    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t4pf9" podStartSLOduration=2.76622404 podStartE2EDuration="22.729694266s" totalImagesPullingTime="19.963470226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.729302893 +0000 UTC m=+23.776498048" watchObservedRunningTime="2026-07-13 18:50:54.729694266 +0000 UTC m=+23.776889425"
Jul 13 18:50:54.753911 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.753844    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r4459" podStartSLOduration=2.786207321 podStartE2EDuration="22.753829251s" totalImagesPullingTime="19.96762193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.753719831 +0000 UTC m=+23.800914987" watchObservedRunningTime="2026-07-13 18:50:54.753829251 +0000 UTC m=+23.801024405"
Jul 13 18:50:54.775782 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.775722    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-48.ec2.internal" podStartSLOduration=21.775706518 podStartE2EDuration="21.775706518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.775480043 +0000 UTC m=+23.822675198" watchObservedRunningTime="2026-07-13 18:50:54.775706518 +0000 UTC m=+23.822901672"
Jul 13 18:50:54.805436 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.805395    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4dq2k" podStartSLOduration=2.8561465249999998 podStartE2EDuration="22.805382905s" totalImagesPullingTime="19.94923638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.804912431 +0000 UTC m=+23.852107585" watchObservedRunningTime="2026-07-13 18:50:54.805382905 +0000 UTC m=+23.852578273"
Jul 13 18:50:54.827382 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:54.827337    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-97w5n" podStartSLOduration=2.886080583 podStartE2EDuration="22.827324228s" totalImagesPullingTime="19.941243645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:54.82732284 +0000 UTC m=+23.874518006" watchObservedRunningTime="2026-07-13 18:50:54.827324228 +0000 UTC m=+23.874519383"
Jul 13 18:50:55.137105 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:55.137072    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:55.137747 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:55.137729    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:55.495859 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:55.495836    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 18:50:55.614149 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:55.614115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86" event={"ID":"b09f0a67-27f1-4660-a46a-c9d30c5952ed","Type":"ContainerStarted","Data":"8239eedf6753e8daf740e78ebd148f786731eda1e14a2648aa3d56f6a92d7ec3"}
Jul 13 18:50:55.615896 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:55.615840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bwhr" event={"ID":"d71dc73c-f80c-4746-887c-4b0d18073d53","Type":"ContainerStarted","Data":"d3586f508883eaa76988f48cbdc96a7cba8948a4041ecd30eb1ec2899c41070f"}
Jul 13 18:50:55.642428 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:55.642337    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6bwhr" podStartSLOduration=3.667063055 podStartE2EDuration="23.642318517s" totalImagesPullingTime="19.975255462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:55.641969235 +0000 UTC m=+24.689164391" watchObservedRunningTime="2026-07-13 18:50:55.642318517 +0000 UTC m=+24.689513676"
Jul 13 18:50:56.410010 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.409861    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T18:50:55.49585236Z","UUID":"ddaf0344-f11e-4ebf-af08-d60956957f25","Handler":null,"Name":"","Endpoint":""}
Jul 13 18:50:56.411621 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.411598    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 18:50:56.411759 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.411632    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 18:50:56.501599 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.501506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:56.501599 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.501548    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:56.501834 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.501506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:56.501834 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:56.501644    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:56.501834 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:56.501773    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:56.501987 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:56.501883    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:56.618548 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:56.618515    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 13 18:50:57.623546 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:57.623197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86" event={"ID":"b09f0a67-27f1-4660-a46a-c9d30c5952ed","Type":"ContainerStarted","Data":"e41b2efce574473295849b3a94656a17ccf78e9c8c1bcccf61ba0c7fa8dea435"}
Jul 13 18:50:57.629943 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:57.629916    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 18:50:57.630329 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:57.630308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"3c42ea6c66fdb4666155e935031fab2d44ab307bbff5ae359743cc3950555b15"}
Jul 13 18:50:58.501019 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:58.500984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:50:58.501232 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:58.500985    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:50:58.501232 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:58.501104    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:50:58.501232 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:58.500998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:50:58.501232 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:58.501184    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:50:58.501432 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:50:58.501314    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:50:59.636436 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.636244    2581 generic.go:356] "Generic (PLEG): container finished" podID="38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e" containerID="fb47fc328cfbe8b0b3694792b5146df167bfb4da4a2c443d4101adb92b22f94d" exitCode=0
Jul 13 18:50:59.637196 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.636276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerDied","Data":"fb47fc328cfbe8b0b3694792b5146df167bfb4da4a2c443d4101adb92b22f94d"}
Jul 13 18:50:59.639925 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.639907    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 18:50:59.640222 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.640201    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"be8bb3385a81aa4a476e670b6f697aa65c1e51e1839640b54495894fc5940d90"}
Jul 13 18:50:59.640473 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.640454    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:59.640541 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.640527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:59.640695 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.640678    2581 scope.go:117] "RemoveContainer" containerID="d1b0a888b588bbd417ad22a7b7e86e1f251d65fa7ccb5c21d6eb49efb8ed3929"
Jul 13 18:50:59.655490 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.655471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:50:59.681200 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.681164    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8g86" podStartSLOduration=9.160189795 podStartE2EDuration="27.681148509s" totalImagesPullingTime="18.520958714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:50:57.641414002 +0000 UTC m=+26.688609181" watchObservedRunningTime="2026-07-13 18:50:59.681148509 +0000 UTC m=+28.728343722"
Jul 13 18:50:59.910660 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.910589    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:50:59.910825 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.910731    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 13 18:50:59.911152 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:50:59.911137    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t4pf9"
Jul 13 18:51:00.500960 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.500929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:00.501143 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:00.501032    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:00.501143 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.501047    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:00.501143 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.500939    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:00.501143 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:00.501137    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:00.501292 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:00.501216    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:00.645525 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.645502    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 18:51:00.645938 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.645885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" event={"ID":"648f005d-b22e-4abd-a731-89b710f8d991","Type":"ContainerStarted","Data":"c5b0324b49c4954a0b369bd5e9bacd148766ccdec02dd46c3e94d6752072d150"}
Jul 13 18:51:00.646042 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.646021    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:51:00.660513 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.660490    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:51:00.681232 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:00.681185    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-frplb" podStartSLOduration=8.655253609 podStartE2EDuration="28.68116849s" totalImagesPullingTime="20.025914881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:00.679285926 +0000 UTC m=+29.726481107" watchObservedRunningTime="2026-07-13 18:51:00.68116849 +0000 UTC m=+29.728363645"
Jul 13 18:51:01.361509 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.361428    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-77cd8"]
Jul 13 18:51:01.361680 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.361572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:01.361748 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:01.361720    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:01.362223 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.362198    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lvmn2"]
Jul 13 18:51:01.362344 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.362289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:01.362423 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:01.362401    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:01.380183 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.380148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5xfpm"]
Jul 13 18:51:01.380297 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.380237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:01.380344 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:01.380329    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:01.649865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.649826    2581 generic.go:356] "Generic (PLEG): container finished" podID="38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e" containerID="2178e7a8c54628c2fcca545057ce01fd23e7690c4e3989a1b3e579fe68958e1f" exitCode=0
Jul 13 18:51:01.650296 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:01.649916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerDied","Data":"2178e7a8c54628c2fcca545057ce01fd23e7690c4e3989a1b3e579fe68958e1f"}
Jul 13 18:51:02.501743 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:02.501707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:02.501908 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:02.501824    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:03.501734 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:03.501543    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:03.502149 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:03.501584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:03.502149 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:03.501813    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:03.502149 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:03.501889    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:03.655437 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:03.655398    2581 generic.go:356] "Generic (PLEG): container finished" podID="38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e" containerID="ab2d08d9b1dfceaf5c225f2b5ed3009e4097b11446b276a85308c56beaf401d3" exitCode=0
Jul 13 18:51:03.655602 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:03.655453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerDied","Data":"ab2d08d9b1dfceaf5c225f2b5ed3009e4097b11446b276a85308c56beaf401d3"}
Jul 13 18:51:04.501090 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:04.501058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:04.501257 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:04.501175    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:04.726385 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:04.726360    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r4459_d4f1d1b0-33f6-4248-b57e-e9af5bd4af07/dns-node-resolver/0.log"
Jul 13 18:51:05.087437 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:05.087405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:05.087613 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.087574    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:51:05.087687 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.087640    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs podName:da08ff36-fa0f-4526-a23e-e2c759749ce6 nodeName:}" failed. No retries permitted until 2026-07-13 18:51:37.08762191 +0000 UTC m=+66.134817066 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs") pod "network-metrics-daemon-5xfpm" (UID: "da08ff36-fa0f-4526-a23e-e2c759749ce6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:51:05.188762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:05.188727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:05.188930 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:05.188778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:05.188930 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.188895    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:51:05.188930 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.188912    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:51:05.188930 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.188923    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x2j65 for pod openshift-network-diagnostics/network-check-target-77cd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:51:05.188930 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.188928    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:51:05.189195 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.188981    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65 podName:a79097ef-170e-4a96-9005-914625cb592b nodeName:}" failed. No retries permitted until 2026-07-13 18:51:37.188962566 +0000 UTC m=+66.236157712 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2j65" (UniqueName: "kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65") pod "network-check-target-77cd8" (UID: "a79097ef-170e-4a96-9005-914625cb592b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:51:05.189195 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.189004    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret podName:27995e44-0dd7-4291-91d5-d0823bd49fb5 nodeName:}" failed. No retries permitted until 2026-07-13 18:51:37.188992974 +0000 UTC m=+66.236188112 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret") pod "global-pull-secret-syncer-lvmn2" (UID: "27995e44-0dd7-4291-91d5-d0823bd49fb5") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:51:05.501439 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:05.501411    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:05.501614 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:05.501472    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97w5n_e291613f-af89-4da4-827c-aaf0c2819787/node-ca/0.log"
Jul 13 18:51:05.501614 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:05.501534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:05.501614 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.501532    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:05.501752 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:05.501645    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:06.501306 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:06.501276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:06.501834 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:06.501398    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:07.501447 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:07.501414    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:07.501880 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:07.501415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:07.501880 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:07.501540    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:07.501880 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:07.501632    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:08.501323 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:08.501297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:08.501475 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:08.501388    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:08.667651 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:08.667622    2581 generic.go:356] "Generic (PLEG): container finished" podID="38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e" containerID="0ee9a3e6d1f610e1ff7043d5c51d257424680f8078679d023208961bced70a1c" exitCode=0
Jul 13 18:51:08.667757 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:08.667667    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerDied","Data":"0ee9a3e6d1f610e1ff7043d5c51d257424680f8078679d023208961bced70a1c"}
Jul 13 18:51:09.501469 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:09.501438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:09.502026 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:09.501573    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:09.502026 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:09.501621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:09.502026 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:09.501694    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:09.672754 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:09.672723    2581 generic.go:356] "Generic (PLEG): container finished" podID="38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e" containerID="70b91ea8b1bd85eb9c6bfe2de5c76b0da281202bc9610c7164fb6a0c75a5c70a" exitCode=0
Jul 13 18:51:09.672912 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:09.672769    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerDied","Data":"70b91ea8b1bd85eb9c6bfe2de5c76b0da281202bc9610c7164fb6a0c75a5c70a"}
Jul 13 18:51:10.501257 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:10.501224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:10.501434 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:10.501334    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:10.679285 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:10.679256    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" event={"ID":"38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e","Type":"ContainerStarted","Data":"4873de040216ac2f433c0e58ef57b9ac0d138248e46ce8f7b01d7c12455c510f"}
Jul 13 18:51:10.707389 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:10.707342    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zcdgx" podStartSLOduration=7.664555526 podStartE2EDuration="38.707327128s" totalImagesPullingTime="31.042771602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:10.707130121 +0000 UTC m=+39.754325289" watchObservedRunningTime="2026-07-13 18:51:10.707327128 +0000 UTC m=+39.754522283"
Jul 13 18:51:11.501916 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:11.501881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:11.502088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:11.501962    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:11.502088 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:11.501968    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:11.502088 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:11.502059    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:12.501524 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:12.501493    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:12.501921 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:12.501616    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:13.501008 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:13.500978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:13.501164 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:13.501100    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:13.501164 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:13.501149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:13.501265 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:13.501223    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:14.501354 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:14.501325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:14.501730 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:14.501426    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:15.501381 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:15.501350    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:15.501782 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:15.501357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:15.501782 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:15.501454    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:15.501782 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:15.501510    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:16.501510 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:16.501476    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:16.501895 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:16.501622    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:17.501496 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:17.501461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:17.501788 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:17.501502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:17.501788 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:17.501619    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:17.501788 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:17.501711    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:18.501193 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:18.501165    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:18.501353 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:18.501269    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:19.501634 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:19.501602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:19.501634 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:19.501628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:19.502015 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:19.501727    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:19.502015 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:19.501823    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:20.501018 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:20.500992    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:20.501170 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:20.501094    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:21.502042 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:21.502012    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:21.502408 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:21.502097    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:21.502408 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:21.502159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:21.502408 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:21.502235    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:22.501609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:22.501581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:22.501779 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:22.501683    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:23.501268 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:23.501239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:23.501268 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:23.501240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:23.501679 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:23.501350    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:23.501679 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:23.501413    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:24.501222 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:24.501192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:24.501441 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:24.501293    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:25.501882 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:25.501847    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:25.502334 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:25.501938    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-77cd8" podUID="a79097ef-170e-4a96-9005-914625cb592b"
Jul 13 18:51:25.502334 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:25.501966    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:25.502334 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:25.502013    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvmn2" podUID="27995e44-0dd7-4291-91d5-d0823bd49fb5"
Jul 13 18:51:26.501675 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.501649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:26.501805 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:26.501743    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xfpm" podUID="da08ff36-fa0f-4526-a23e-e2c759749ce6"
Jul 13 18:51:26.749254 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.749233    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-48.ec2.internal" event="NodeReady"
Jul 13 18:51:26.749636 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.749318    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 18:51:26.815666 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.815637    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h7v5n"]
Jul 13 18:51:26.839952 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.839930    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9t4gz"]
Jul 13 18:51:26.840084 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.840069    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:26.842115 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.842092    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 18:51:26.842706 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.842684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 18:51:26.843089 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.843074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pslxn\""
Jul 13 18:51:26.864188 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.864111    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9t4gz"]
Jul 13 18:51:26.864264 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.864219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7v5n"]
Jul 13 18:51:26.864264 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.864171    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:26.867679 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.867663    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kk5wl\""
Jul 13 18:51:26.868090 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.868073    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 18:51:26.868180 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.868165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 18:51:26.868289 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.868275    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 18:51:26.932660 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.932639    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bkttc"]
Jul 13 18:51:26.941247 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.941222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6083f9c2-e560-42fb-a5c5-52cb000f7920-config-volume\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:26.941317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.941256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6083f9c2-e560-42fb-a5c5-52cb000f7920-tmp-dir\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:26.941317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.941275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67d203d3-7769-4e27-b2f8-b928e9094e72-cert\") pod \"ingress-canary-9t4gz\" (UID: \"67d203d3-7769-4e27-b2f8-b928e9094e72\") " pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:26.941317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.941292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6083f9c2-e560-42fb-a5c5-52cb000f7920-metrics-tls\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:26.941317 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.941312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtt9z\" (UniqueName: \"kubernetes.io/projected/6083f9c2-e560-42fb-a5c5-52cb000f7920-kube-api-access-dtt9z\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:26.941437 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.941367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5tzg\" (UniqueName: \"kubernetes.io/projected/67d203d3-7769-4e27-b2f8-b928e9094e72-kube-api-access-k5tzg\") pod \"ingress-canary-9t4gz\" (UID: \"67d203d3-7769-4e27-b2f8-b928e9094e72\") " pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:26.956746 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.956728    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:26.961680 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.961659    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkttc"]
Jul 13 18:51:26.962099 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.962072    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 18:51:26.962385 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.962370    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 18:51:26.962651 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.962630    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 18:51:26.962952 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.962932    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kqgjs\""
Jul 13 18:51:26.963568 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:26.963535    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 18:51:27.041933 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.041870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-data-volume\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.041933 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.041923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6083f9c2-e560-42fb-a5c5-52cb000f7920-config-volume\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.042112 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.041957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6083f9c2-e560-42fb-a5c5-52cb000f7920-tmp-dir\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.042112 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.041980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-crio-socket\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.042112 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67d203d3-7769-4e27-b2f8-b928e9094e72-cert\") pod \"ingress-canary-9t4gz\" (UID: \"67d203d3-7769-4e27-b2f8-b928e9094e72\") " pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:27.042112 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btrd4\" (UniqueName: \"kubernetes.io/projected/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-kube-api-access-btrd4\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.042112 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6083f9c2-e560-42fb-a5c5-52cb000f7920-metrics-tls\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.042335 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtt9z\" (UniqueName: \"kubernetes.io/projected/6083f9c2-e560-42fb-a5c5-52cb000f7920-kube-api-access-dtt9z\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.042335 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.042335 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5tzg\" (UniqueName: \"kubernetes.io/projected/67d203d3-7769-4e27-b2f8-b928e9094e72-kube-api-access-k5tzg\") pod \"ingress-canary-9t4gz\" (UID: \"67d203d3-7769-4e27-b2f8-b928e9094e72\") " pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:27.042335 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.042335 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6083f9c2-e560-42fb-a5c5-52cb000f7920-tmp-dir\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.042540 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.042523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6083f9c2-e560-42fb-a5c5-52cb000f7920-config-volume\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.045992 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.045974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67d203d3-7769-4e27-b2f8-b928e9094e72-cert\") pod \"ingress-canary-9t4gz\" (UID: \"67d203d3-7769-4e27-b2f8-b928e9094e72\") " pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:27.046068 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.045978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6083f9c2-e560-42fb-a5c5-52cb000f7920-metrics-tls\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.052077 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.052051    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtt9z\" (UniqueName: \"kubernetes.io/projected/6083f9c2-e560-42fb-a5c5-52cb000f7920-kube-api-access-dtt9z\") pod \"dns-default-h7v5n\" (UID: \"6083f9c2-e560-42fb-a5c5-52cb000f7920\") " pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.052353 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.052337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5tzg\" (UniqueName: \"kubernetes.io/projected/67d203d3-7769-4e27-b2f8-b928e9094e72-kube-api-access-k5tzg\") pod \"ingress-canary-9t4gz\" (UID: \"67d203d3-7769-4e27-b2f8-b928e9094e72\") " pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:27.143459 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-crio-socket\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.143459 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btrd4\" (UniqueName: \"kubernetes.io/projected/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-kube-api-access-btrd4\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.143661 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.143661 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.143661 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-data-volume\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.143661 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143603    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-crio-socket\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.143849 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.143781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-data-volume\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.144112 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.144090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.145547 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.145530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.149563 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.149535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:27.155531 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.155511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btrd4\" (UniqueName: \"kubernetes.io/projected/8c0c6a97-5c16-4552-8260-e3b3eaa173e1-kube-api-access-btrd4\") pod \"insights-runtime-extractor-bkttc\" (UID: \"8c0c6a97-5c16-4552-8260-e3b3eaa173e1\") " pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.172455 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.172435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9t4gz"
Jul 13 18:51:27.266100 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.266069    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkttc"
Jul 13 18:51:27.367752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.361854    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9t4gz"]
Jul 13 18:51:27.393127 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.393102    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7v5n"]
Jul 13 18:51:27.396745 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:27.396717    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6083f9c2_e560_42fb_a5c5_52cb000f7920.slice/crio-f055f93ac96b1afc759e90122db352f81eb00ee2c772b5c5cbd9224ea159377b WatchSource:0}: Error finding container f055f93ac96b1afc759e90122db352f81eb00ee2c772b5c5cbd9224ea159377b: Status 404 returned error can't find the container with id f055f93ac96b1afc759e90122db352f81eb00ee2c772b5c5cbd9224ea159377b
Jul 13 18:51:27.447584 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.447538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkttc"]
Jul 13 18:51:27.451011 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:27.450989    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0c6a97_5c16_4552_8260_e3b3eaa173e1.slice/crio-610eab131fdefbbd93f4404c3cd134fb9aac4ddc4a848ad83949264f362f022a WatchSource:0}: Error finding container 610eab131fdefbbd93f4404c3cd134fb9aac4ddc4a848ad83949264f362f022a: Status 404 returned error can't find the container with id 610eab131fdefbbd93f4404c3cd134fb9aac4ddc4a848ad83949264f362f022a
Jul 13 18:51:27.501703 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.501679    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:27.501872 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.501849    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:27.506355 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.506336    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 18:51:27.506447 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.506395    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 18:51:27.506500 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.506463    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 18:51:27.506952 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.506939    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2n8qw\""
Jul 13 18:51:27.716063 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.716033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9t4gz" event={"ID":"67d203d3-7769-4e27-b2f8-b928e9094e72","Type":"ContainerStarted","Data":"b9f1f39e87662c45b5856c033bcd4ada74519706ecb4bcf3dfe30ab073023b56"}
Jul 13 18:51:27.716995 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.716972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7v5n" event={"ID":"6083f9c2-e560-42fb-a5c5-52cb000f7920","Type":"ContainerStarted","Data":"f055f93ac96b1afc759e90122db352f81eb00ee2c772b5c5cbd9224ea159377b"}
Jul 13 18:51:27.717904 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:27.717886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkttc" event={"ID":"8c0c6a97-5c16-4552-8260-e3b3eaa173e1","Type":"ContainerStarted","Data":"610eab131fdefbbd93f4404c3cd134fb9aac4ddc4a848ad83949264f362f022a"}
Jul 13 18:51:28.472638 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.472604    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-789c65455c-b5h9j"]
Jul 13 18:51:28.487620 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.487595    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.490746 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.490721    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 18:51:28.490746 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.490725    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 18:51:28.491405 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.491384    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 18:51:28.491505 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.491458    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 18:51:28.491575 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.491395    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 18:51:28.492100 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.491809    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vbvqm\""
Jul 13 18:51:28.492100 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.491893    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 18:51:28.492100 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.491814    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 18:51:28.499279 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.499259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-789c65455c-b5h9j"]
Jul 13 18:51:28.501764 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.501746    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:28.507158 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.507141    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qx8g5\""
Jul 13 18:51:28.508048 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.508030    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 18:51:28.554963 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.554936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-config\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.555095 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.554972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfp24\" (UniqueName: \"kubernetes.io/projected/2722a629-10f5-4ecb-9e4d-b7179d9a124f-kube-api-access-kfp24\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.555095 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.555005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-service-ca\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.555205 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.555128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-oauth-config\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.555205 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.555169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-oauth-serving-cert\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.555205 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.555201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-serving-cert\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.656234 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656210    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-oauth-config\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.656348 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-oauth-serving-cert\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.656348 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-serving-cert\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.656451 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-config\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.656451 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfp24\" (UniqueName: \"kubernetes.io/projected/2722a629-10f5-4ecb-9e4d-b7179d9a124f-kube-api-access-kfp24\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.656547 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-service-ca\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.657001 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.656975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-oauth-serving-cert\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.657087 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.657035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-config\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.657087 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.657045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-service-ca\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.658619 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.658598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-serving-cert\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.658990 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.658974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-oauth-config\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.665256 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.665232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfp24\" (UniqueName: \"kubernetes.io/projected/2722a629-10f5-4ecb-9e4d-b7179d9a124f-kube-api-access-kfp24\") pod \"console-789c65455c-b5h9j\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") " pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:28.798636 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:28.798573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:29.735989 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:29.735965    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-789c65455c-b5h9j"]
Jul 13 18:51:29.900684 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:29.900642    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2722a629_10f5_4ecb_9e4d_b7179d9a124f.slice/crio-35ea2a1cb7f42a7eab7e94d1177491cc26faba52309fc18068fec1d7e47008c6 WatchSource:0}: Error finding container 35ea2a1cb7f42a7eab7e94d1177491cc26faba52309fc18068fec1d7e47008c6: Status 404 returned error can't find the container with id 35ea2a1cb7f42a7eab7e94d1177491cc26faba52309fc18068fec1d7e47008c6
Jul 13 18:51:30.559758 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.559517    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"]
Jul 13 18:51:30.561838 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.561782    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:30.565500 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.565013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-p5p9p\""
Jul 13 18:51:30.565500 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.565254    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 13 18:51:30.581826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.581772    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"]
Jul 13 18:51:30.675148 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.672697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-n8gs4\" (UID: \"c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:30.728885 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.728589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9t4gz" event={"ID":"67d203d3-7769-4e27-b2f8-b928e9094e72","Type":"ContainerStarted","Data":"c10f5c684395b48508256be250a81cfeb8c2372435591eedb2a37f77f24f89f9"}
Jul 13 18:51:30.729997 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.729968    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-789c65455c-b5h9j" event={"ID":"2722a629-10f5-4ecb-9e4d-b7179d9a124f","Type":"ContainerStarted","Data":"35ea2a1cb7f42a7eab7e94d1177491cc26faba52309fc18068fec1d7e47008c6"}
Jul 13 18:51:30.745022 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.744976    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9t4gz" podStartSLOduration=2.182402934 podStartE2EDuration="4.744962193s" totalImagesPullingTime="2.562559259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:30.744044817 +0000 UTC m=+59.791239973" watchObservedRunningTime="2026-07-13 18:51:30.744962193 +0000 UTC m=+59.792157348"
Jul 13 18:51:30.773144 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.773112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-n8gs4\" (UID: \"c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:30.776117 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.776087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-n8gs4\" (UID: \"c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:30.874715 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:30.874634    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:31.028445 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:31.028367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"]
Jul 13 18:51:31.505981 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:31.505939    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ab8b22_9042_40b9_bcd7_8bb1e134c8a7.slice/crio-dcd9f7d59ab671ed962101501a1749c8068ee96cef8d53eebe80ceab90ae3414 WatchSource:0}: Error finding container dcd9f7d59ab671ed962101501a1749c8068ee96cef8d53eebe80ceab90ae3414: Status 404 returned error can't find the container with id dcd9f7d59ab671ed962101501a1749c8068ee96cef8d53eebe80ceab90ae3414
Jul 13 18:51:31.734700 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:31.734634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4" event={"ID":"c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7","Type":"ContainerStarted","Data":"dcd9f7d59ab671ed962101501a1749c8068ee96cef8d53eebe80ceab90ae3414"}
Jul 13 18:51:32.662636 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:32.662610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-frplb"
Jul 13 18:51:35.745891 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.745864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-789c65455c-b5h9j" event={"ID":"2722a629-10f5-4ecb-9e4d-b7179d9a124f","Type":"ContainerStarted","Data":"726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991"}
Jul 13 18:51:35.747390 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.747366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4" event={"ID":"c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7","Type":"ContainerStarted","Data":"655b04f903a7cf9afc2211d98748c0f118dd490e239c1e7daa3af43b3aea4914"}
Jul 13 18:51:35.747598 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.747568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:35.749406 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.749384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkttc" event={"ID":"8c0c6a97-5c16-4552-8260-e3b3eaa173e1","Type":"ContainerStarted","Data":"3acb1a0a1f32e8af73919222388fb73c436a05acd3f262cf8f7a1d0819a10b3a"}
Jul 13 18:51:35.751333 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.751285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7v5n" event={"ID":"6083f9c2-e560-42fb-a5c5-52cb000f7920","Type":"ContainerStarted","Data":"441000a1a5d3df9e73efd73a04e84f45f18e59a010fda999c8533e3fa49987c6"}
Jul 13 18:51:35.751441 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.751354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7v5n" event={"ID":"6083f9c2-e560-42fb-a5c5-52cb000f7920","Type":"ContainerStarted","Data":"6927a3517b2bddcc167e63a1661505eed671a9b892ed1511159d15bbaea1275d"}
Jul 13 18:51:35.751441 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.751435    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:35.753012 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.752986    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4"
Jul 13 18:51:35.771444 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.771407    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-789c65455c-b5h9j" podStartSLOduration=2.616918334 podStartE2EDuration="7.771395845s" totalImagesPullingTime="5.154477511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:35.770684331 +0000 UTC m=+64.817879486" watchObservedRunningTime="2026-07-13 18:51:35.771395845 +0000 UTC m=+64.818590999"
Jul 13 18:51:35.791934 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.791894    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-n8gs4" podStartSLOduration=2.184728767 podStartE2EDuration="5.791882113s" totalImagesPullingTime="3.607153346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:35.791715754 +0000 UTC m=+64.838910909" watchObservedRunningTime="2026-07-13 18:51:35.791882113 +0000 UTC m=+64.839077268"
Jul 13 18:51:35.816688 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:35.816654    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h7v5n" podStartSLOduration=2.140227653 podStartE2EDuration="9.816640957s" totalImagesPullingTime="7.676413304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:35.815777477 +0000 UTC m=+64.862972633" watchObservedRunningTime="2026-07-13 18:51:35.816640957 +0000 UTC m=+64.863836112"
Jul 13 18:51:36.734771 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.734738    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-b9xg5"]
Jul 13 18:51:36.738452 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.738022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.741487 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.741461    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 13 18:51:36.742115 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.742090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 18:51:36.742265 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.742236    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 18:51:36.742392 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.742331    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bfjsb\""
Jul 13 18:51:36.742895 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.742872    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 18:51:36.746832 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.746812    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 13 18:51:36.756968 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.756942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkttc" event={"ID":"8c0c6a97-5c16-4552-8260-e3b3eaa173e1","Type":"ContainerStarted","Data":"b8bcc6d3327ef0b5ba98cdf3cb818458d383bae22c4031baa129b97133e1f222"}
Jul 13 18:51:36.762054 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.762033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-b9xg5"]
Jul 13 18:51:36.821274 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.821241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d00969cc-c4a3-444f-bad3-2cfe468feed2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.821402 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.821284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00969cc-c4a3-444f-bad3-2cfe468feed2-metrics-client-ca\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.821402 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.821347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2szw\" (UniqueName: \"kubernetes.io/projected/d00969cc-c4a3-444f-bad3-2cfe468feed2-kube-api-access-b2szw\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.821513 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.821467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d00969cc-c4a3-444f-bad3-2cfe468feed2-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.922411 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.922379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d00969cc-c4a3-444f-bad3-2cfe468feed2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.922594 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.922416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00969cc-c4a3-444f-bad3-2cfe468feed2-metrics-client-ca\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.922594 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.922451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2szw\" (UniqueName: \"kubernetes.io/projected/d00969cc-c4a3-444f-bad3-2cfe468feed2-kube-api-access-b2szw\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.922594 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.922484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d00969cc-c4a3-444f-bad3-2cfe468feed2-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.923315 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.923265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00969cc-c4a3-444f-bad3-2cfe468feed2-metrics-client-ca\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.925073 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.925050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d00969cc-c4a3-444f-bad3-2cfe468feed2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.925278 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.925260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d00969cc-c4a3-444f-bad3-2cfe468feed2-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:36.936834 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:36.936815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2szw\" (UniqueName: \"kubernetes.io/projected/d00969cc-c4a3-444f-bad3-2cfe468feed2-kube-api-access-b2szw\") pod \"prometheus-operator-64785b6675-b9xg5\" (UID: \"d00969cc-c4a3-444f-bad3-2cfe468feed2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:37.049855 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.049783    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5"
Jul 13 18:51:37.124765 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.124731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:37.127088 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.127067    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 18:51:37.138035 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.138012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da08ff36-fa0f-4526-a23e-e2c759749ce6-metrics-certs\") pod \"network-metrics-daemon-5xfpm\" (UID: \"da08ff36-fa0f-4526-a23e-e2c759749ce6\") " pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:37.186865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.186839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-b9xg5"]
Jul 13 18:51:37.189386 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:37.189361    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00969cc_c4a3_444f_bad3_2cfe468feed2.slice/crio-f18407223e2746e4ad88cc0a6785122dc8fa118642c57ab91fd2f21312e38f52 WatchSource:0}: Error finding container f18407223e2746e4ad88cc0a6785122dc8fa118642c57ab91fd2f21312e38f52: Status 404 returned error can't find the container with id f18407223e2746e4ad88cc0a6785122dc8fa118642c57ab91fd2f21312e38f52
Jul 13 18:51:37.214054 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.214035    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qx8g5\""
Jul 13 18:51:37.222772 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.222752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xfpm"
Jul 13 18:51:37.225175 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.225055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:37.225175 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.225106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:37.227069 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.227049    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 18:51:37.227163 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.227051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 18:51:37.238147 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.238124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 18:51:37.238511 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.238488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27995e44-0dd7-4291-91d5-d0823bd49fb5-original-pull-secret\") pod \"global-pull-secret-syncer-lvmn2\" (UID: \"27995e44-0dd7-4291-91d5-d0823bd49fb5\") " pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:37.249106 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.249084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2j65\" (UniqueName: \"kubernetes.io/projected/a79097ef-170e-4a96-9005-914625cb592b-kube-api-access-x2j65\") pod \"network-check-target-77cd8\" (UID: \"a79097ef-170e-4a96-9005-914625cb592b\") " pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:37.362285 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.362253    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5xfpm"]
Jul 13 18:51:37.365769 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:37.365740    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda08ff36_fa0f_4526_a23e_e2c759749ce6.slice/crio-61f9de9af497eeab02008427fd693fa9ffc815a329039d3934fb234a666e7172 WatchSource:0}: Error finding container 61f9de9af497eeab02008427fd693fa9ffc815a329039d3934fb234a666e7172: Status 404 returned error can't find the container with id 61f9de9af497eeab02008427fd693fa9ffc815a329039d3934fb234a666e7172
Jul 13 18:51:37.414172 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.414148    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2n8qw\""
Jul 13 18:51:37.416004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.415981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvmn2"
Jul 13 18:51:37.422024 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.422005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:37.559388 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.559353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-77cd8"]
Jul 13 18:51:37.577025 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.576996    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lvmn2"]
Jul 13 18:51:37.761148 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.761112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xfpm" event={"ID":"da08ff36-fa0f-4526-a23e-e2c759749ce6","Type":"ContainerStarted","Data":"61f9de9af497eeab02008427fd693fa9ffc815a329039d3934fb234a666e7172"}
Jul 13 18:51:37.762436 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:37.762409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5" event={"ID":"d00969cc-c4a3-444f-bad3-2cfe468feed2","Type":"ContainerStarted","Data":"f18407223e2746e4ad88cc0a6785122dc8fa118642c57ab91fd2f21312e38f52"}
Jul 13 18:51:38.412146 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.412113    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c586ccf97-q8h5b"]
Jul 13 18:51:38.421227 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.421200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.424188 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.424165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c586ccf97-q8h5b"]
Jul 13 18:51:38.429043 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.429015    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 18:51:38.535356 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-trusted-ca-bundle\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.535537 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-service-ca\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.535537 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-config\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.535537 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-serving-cert\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.535537 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-oauth-config\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.535537 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7hp9\" (UniqueName: \"kubernetes.io/projected/a8c6589a-ade4-413c-828a-93d7ff21cf13-kube-api-access-c7hp9\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.535751 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.535550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-oauth-serving-cert\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.575565 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:38.575519    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27995e44_0dd7_4291_91d5_d0823bd49fb5.slice/crio-100ad15acfe107cf6c512431224d6aadaac7c4a97d65c4d6355f8c757b417aa2 WatchSource:0}: Error finding container 100ad15acfe107cf6c512431224d6aadaac7c4a97d65c4d6355f8c757b417aa2: Status 404 returned error can't find the container with id 100ad15acfe107cf6c512431224d6aadaac7c4a97d65c4d6355f8c757b417aa2
Jul 13 18:51:38.576279 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:38.576068    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79097ef_170e_4a96_9005_914625cb592b.slice/crio-d7cc4a0211e94ec1e1546755183d8079a0dca67fea9d82c902a2c997cfed9ff1 WatchSource:0}: Error finding container d7cc4a0211e94ec1e1546755183d8079a0dca67fea9d82c902a2c997cfed9ff1: Status 404 returned error can't find the container with id d7cc4a0211e94ec1e1546755183d8079a0dca67fea9d82c902a2c997cfed9ff1
Jul 13 18:51:38.636392 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7hp9\" (UniqueName: \"kubernetes.io/projected/a8c6589a-ade4-413c-828a-93d7ff21cf13-kube-api-access-c7hp9\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.636486 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-oauth-serving-cert\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.636486 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-trusted-ca-bundle\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.636595 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-service-ca\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.636646 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-config\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.636686 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-serving-cert\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.636738 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.636709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-oauth-config\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.637219 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.637195    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-oauth-serving-cert\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.637343 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.637320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-service-ca\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.637422 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.637324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-config\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.637422 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.637376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-trusted-ca-bundle\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.639093 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.639069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-serving-cert\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.639426 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.639409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-oauth-config\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.645920 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.645899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7hp9\" (UniqueName: \"kubernetes.io/projected/a8c6589a-ade4-413c-828a-93d7ff21cf13-kube-api-access-c7hp9\") pod \"console-c586ccf97-q8h5b\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") " pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.732932 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.732713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:38.770521 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.770477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-77cd8" event={"ID":"a79097ef-170e-4a96-9005-914625cb592b","Type":"ContainerStarted","Data":"d7cc4a0211e94ec1e1546755183d8079a0dca67fea9d82c902a2c997cfed9ff1"}
Jul 13 18:51:38.774154 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.774120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lvmn2" event={"ID":"27995e44-0dd7-4291-91d5-d0823bd49fb5","Type":"ContainerStarted","Data":"100ad15acfe107cf6c512431224d6aadaac7c4a97d65c4d6355f8c757b417aa2"}
Jul 13 18:51:38.778087 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.778055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkttc" event={"ID":"8c0c6a97-5c16-4552-8260-e3b3eaa173e1","Type":"ContainerStarted","Data":"809563465c317c017904011a70a835bfb931d136277b182625757e1f08155512"}
Jul 13 18:51:38.798008 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.797329    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bkttc" podStartSLOduration=1.787585602 podStartE2EDuration="12.797307977s" totalImagesPullingTime="11.009722375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:51:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:38.796278815 +0000 UTC m=+67.843473974" watchObservedRunningTime="2026-07-13 18:51:38.797307977 +0000 UTC m=+67.844503137"
Jul 13 18:51:38.799239 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.798908    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:38.799396 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.799354    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:38.807080 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.807059    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:38.885710 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:38.885678    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c586ccf97-q8h5b"]
Jul 13 18:51:38.890052 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:38.890019    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c6589a_ade4_413c_828a_93d7ff21cf13.slice/crio-f26ee898202eb2e8639ac1e9c8418528abe117f5dabd32bafbd66c7354f2030f WatchSource:0}: Error finding container f26ee898202eb2e8639ac1e9c8418528abe117f5dabd32bafbd66c7354f2030f: Status 404 returned error can't find the container with id f26ee898202eb2e8639ac1e9c8418528abe117f5dabd32bafbd66c7354f2030f
Jul 13 18:51:39.784731 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.784694    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5" event={"ID":"d00969cc-c4a3-444f-bad3-2cfe468feed2","Type":"ContainerStarted","Data":"86954e25956bd188331261e1482577fade5684897ea505eaf3322fdd48e99430"}
Jul 13 18:51:39.785159 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.784758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5" event={"ID":"d00969cc-c4a3-444f-bad3-2cfe468feed2","Type":"ContainerStarted","Data":"84f8230786199553bb97e579f299c7ef18246709fc308b87c31d1e5a76ecdfd4"}
Jul 13 18:51:39.786689 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.786657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c586ccf97-q8h5b" event={"ID":"a8c6589a-ade4-413c-828a-93d7ff21cf13","Type":"ContainerStarted","Data":"bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840"}
Jul 13 18:51:39.786809 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.786708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c586ccf97-q8h5b" event={"ID":"a8c6589a-ade4-413c-828a-93d7ff21cf13","Type":"ContainerStarted","Data":"f26ee898202eb2e8639ac1e9c8418528abe117f5dabd32bafbd66c7354f2030f"}
Jul 13 18:51:39.789693 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.789611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xfpm" event={"ID":"da08ff36-fa0f-4526-a23e-e2c759749ce6","Type":"ContainerStarted","Data":"9201b3c80827ff0adac2a01b2b45fbbe015cc65685b0aec8cc69de28740a8480"}
Jul 13 18:51:39.789693 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.789659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xfpm" event={"ID":"da08ff36-fa0f-4526-a23e-e2c759749ce6","Type":"ContainerStarted","Data":"cd318065c22e3928165d8f40b0b66a2600defd074c984a67efa4d3f4ede609a2"}
Jul 13 18:51:39.795364 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.795344    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:51:39.819284 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.819208    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-b9xg5" podStartSLOduration=1.446295893 podStartE2EDuration="3.819192599s" totalImagesPullingTime="2.372896706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:39.818329516 +0000 UTC m=+68.865524670" watchObservedRunningTime="2026-07-13 18:51:39.819192599 +0000 UTC m=+68.866387745"
Jul 13 18:51:39.849252 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.849140    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5xfpm" podStartSLOduration=65.653821111 podStartE2EDuration="1m7.849100463s" totalImagesPullingTime="2.195279352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:39.846906859 +0000 UTC m=+68.894102040" watchObservedRunningTime="2026-07-13 18:51:39.849100463 +0000 UTC m=+68.896295619"
Jul 13 18:51:39.913465 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:39.912616    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c586ccf97-q8h5b" podStartSLOduration=1.912592173 podStartE2EDuration="1.912592173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:39.90959069 +0000 UTC m=+68.956785848" watchObservedRunningTime="2026-07-13 18:51:39.912592173 +0000 UTC m=+68.959787328"
Jul 13 18:51:42.150252 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.150213    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"]
Jul 13 18:51:42.164255 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.164232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.168039 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.167987    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 13 18:51:42.168155 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.168052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-t6vqt\""
Jul 13 18:51:42.168155 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.168059    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 13 18:51:42.184223 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.184202    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"]
Jul 13 18:51:42.212667 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.212637    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-npsqx"]
Jul 13 18:51:42.234238 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.234206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.236899 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.236864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 18:51:42.237061 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.237009    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 18:51:42.237198 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.237181    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 18:51:42.239345 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.239325    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5qsc5\""
Jul 13 18:51:42.240731 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.240711    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"]
Jul 13 18:51:42.259691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.259671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.263305 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.263204    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 13 18:51:42.263853 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.263832    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 13 18:51:42.264966 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.264944    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"]
Jul 13 18:51:42.265305 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.265291    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tsvcq\""
Jul 13 18:51:42.265393 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.265349    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 13 18:51:42.267159 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.267138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9654d578-32ab-4a63-b69c-675cd9c1a38e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.267267 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.267173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9654d578-32ab-4a63-b69c-675cd9c1a38e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.267267 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.267204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9654d578-32ab-4a63-b69c-675cd9c1a38e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.267267 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.267233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9km\" (UniqueName: \"kubernetes.io/projected/9654d578-32ab-4a63-b69c-675cd9c1a38e-kube-api-access-cl9km\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.368237 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9654d578-32ab-4a63-b69c-675cd9c1a38e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.368237 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.368471 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-textfile\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.368471 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.368471 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9654d578-32ab-4a63-b69c-675cd9c1a38e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.368471 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-sys\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.368471 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-tls\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.368752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.368752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9km\" (UniqueName: \"kubernetes.io/projected/9654d578-32ab-4a63-b69c-675cd9c1a38e-kube-api-access-cl9km\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.368752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.368752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lbjk\" (UniqueName: \"kubernetes.io/projected/7da60cc6-7300-4da3-9c10-dba51771c6dd-kube-api-access-5lbjk\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.368752 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-accelerators-collector-config\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-root\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-wtmp\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7da60cc6-7300-4da3-9c10-dba51771c6dd-metrics-client-ca\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368849    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368895    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9654d578-32ab-4a63-b69c-675cd9c1a38e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.369004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.368960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9gb\" (UniqueName: \"kubernetes.io/projected/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-api-access-fj9gb\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.369255 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.369093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9654d578-32ab-4a63-b69c-675cd9c1a38e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.370977 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.370948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9654d578-32ab-4a63-b69c-675cd9c1a38e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.371306 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.371283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9654d578-32ab-4a63-b69c-675cd9c1a38e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.387162 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.387141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9km\" (UniqueName: \"kubernetes.io/projected/9654d578-32ab-4a63-b69c-675cd9c1a38e-kube-api-access-cl9km\") pod \"openshift-state-metrics-84bd68b896-csgmg\" (UID: \"9654d578-32ab-4a63-b69c-675cd9c1a38e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.470395 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.470395 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9gb\" (UniqueName: \"kubernetes.io/projected/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-api-access-fj9gb\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.470395 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-textfile\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-sys\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-tls\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lbjk\" (UniqueName: \"kubernetes.io/projected/7da60cc6-7300-4da3-9c10-dba51771c6dd-kube-api-access-5lbjk\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-accelerators-collector-config\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.470691 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-root\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-wtmp\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7da60cc6-7300-4da3-9c10-dba51771c6dd-metrics-client-ca\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-textfile\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-root\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.470917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.471122 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.471019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-wtmp\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471460 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.471308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da60cc6-7300-4da3-9c10-dba51771c6dd-sys\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471460 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.471349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-accelerators-collector-config\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471460 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.471437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7da60cc6-7300-4da3-9c10-dba51771c6dd-metrics-client-ca\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.471633 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.471533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.472015 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.471971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.473428 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.473401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.473524 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.473422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.473524 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.473481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.473796 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.473777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7da60cc6-7300-4da3-9c10-dba51771c6dd-node-exporter-tls\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.474612 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.474594    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"
Jul 13 18:51:42.494689 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.494670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lbjk\" (UniqueName: \"kubernetes.io/projected/7da60cc6-7300-4da3-9c10-dba51771c6dd-kube-api-access-5lbjk\") pod \"node-exporter-npsqx\" (UID: \"7da60cc6-7300-4da3-9c10-dba51771c6dd\") " pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.494831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.494794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9gb\" (UniqueName: \"kubernetes.io/projected/f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6-kube-api-access-fj9gb\") pod \"kube-state-metrics-785778ff74-nz8bs\" (UID: \"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:42.545659 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.545628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-npsqx"
Jul 13 18:51:42.571629 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:42.571596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"
Jul 13 18:51:43.297033 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.296998    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 18:51:43.300258 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.300236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.303245 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.303218    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 18:51:43.305913 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.305602    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 18:51:43.305913 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.305730    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 18:51:43.305913 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.305811    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 18:51:43.306134 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.305931    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 18:51:43.306134 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.306097    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 18:51:43.306241 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.306207    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 18:51:43.306295 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.306209    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zffld\""
Jul 13 18:51:43.306890 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.306871    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 18:51:43.306991 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.306876    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 18:51:43.320039 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.320012    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 18:51:43.378362 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffpxw\" (UniqueName: \"kubernetes.io/projected/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-kube-api-access-ffpxw\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378519 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378519 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378519 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-web-config\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378538    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-config-volume\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378938 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378938 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-config-out\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378938 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378784    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.378938 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.378821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.479468 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.479418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.479679 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.479479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.479679 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.479640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.479795 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.479696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.479795 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.479732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-config-out\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.479795 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.479760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpxw\" (UniqueName: \"kubernetes.io/projected/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-kube-api-access-ffpxw\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-web-config\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-config-volume\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.480591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481366 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.481361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.481900 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:51:43.481876    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-trusted-ca-bundle podName:af340ad3-9c0f-4ec5-8318-1cb719fa89ec nodeName:}" failed. No retries permitted until 2026-07-13 18:51:43.981853243 +0000 UTC m=+73.029048383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "af340ad3-9c0f-4ec5-8318-1cb719fa89ec") : configmap references non-existent config key: ca-bundle.crt
Jul 13 18:51:43.482524 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.482479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.483407 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.483357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.483840 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.483785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-config-out\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.484046 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.484008    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.484485 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.484463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.484772 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.484749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.484826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.484781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-config-volume\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.484826 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.484814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.485661 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.485641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-web-config\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.496287 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.496244    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffpxw\" (UniqueName: \"kubernetes.io/projected/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-kube-api-access-ffpxw\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.720764 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:43.720729    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da60cc6_7300_4da3_9c10_dba51771c6dd.slice/crio-2699fe8a7d67f63147a421abaa7de5faa57bf2c466c35720e9592fc62cb564ab WatchSource:0}: Error finding container 2699fe8a7d67f63147a421abaa7de5faa57bf2c466c35720e9592fc62cb564ab: Status 404 returned error can't find the container with id 2699fe8a7d67f63147a421abaa7de5faa57bf2c466c35720e9592fc62cb564ab
Jul 13 18:51:43.829620 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.829536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npsqx" event={"ID":"7da60cc6-7300-4da3-9c10-dba51771c6dd","Type":"ContainerStarted","Data":"2699fe8a7d67f63147a421abaa7de5faa57bf2c466c35720e9592fc62cb564ab"}
Jul 13 18:51:43.885110 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.885026    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nz8bs"]
Jul 13 18:51:43.888224 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:43.888159    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f788a5_4fe2_4e47_81a1_1ccf6bd484c6.slice/crio-3559c930c307598c3ab7b1cd8b68001a3890741e85c8d086f4ce57df4bdbfc00 WatchSource:0}: Error finding container 3559c930c307598c3ab7b1cd8b68001a3890741e85c8d086f4ce57df4bdbfc00: Status 404 returned error can't find the container with id 3559c930c307598c3ab7b1cd8b68001a3890741e85c8d086f4ce57df4bdbfc00
Jul 13 18:51:43.903782 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.903758    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg"]
Jul 13 18:51:43.906434 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:43.906410    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9654d578_32ab_4a63_b69c_675cd9c1a38e.slice/crio-a90bef9b7cad05f2bf3216d963d6c19bc16eb5ccffe910f58347685bd0ba1eed WatchSource:0}: Error finding container a90bef9b7cad05f2bf3216d963d6c19bc16eb5ccffe910f58347685bd0ba1eed: Status 404 returned error can't find the container with id a90bef9b7cad05f2bf3216d963d6c19bc16eb5ccffe910f58347685bd0ba1eed
Jul 13 18:51:43.986102 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.986076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:43.987016 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:43.986996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af340ad3-9c0f-4ec5-8318-1cb719fa89ec-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af340ad3-9c0f-4ec5-8318-1cb719fa89ec\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:44.212706 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.212666    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 18:51:44.298759 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.298734    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"]
Jul 13 18:51:44.303065 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.303046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.316319 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.316296    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 13 18:51:44.316467 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.316358    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 13 18:51:44.316467 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.316411    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 13 18:51:44.317642 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.317624    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 13 18:51:44.317770 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.317754    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 13 18:51:44.317836 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.317823    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rv85v\""
Jul 13 18:51:44.317905 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.317889    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-226c2hv0qhhdg\""
Jul 13 18:51:44.352969 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.352939    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"]
Jul 13 18:51:44.371851 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.371770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 18:51:44.374294 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:44.374269    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf340ad3_9c0f_4ec5_8318_1cb719fa89ec.slice/crio-2ced6948a08d4385cf72af97da34d0b1b4af259ae91904deb0d27c779314b36c WatchSource:0}: Error finding container 2ced6948a08d4385cf72af97da34d0b1b4af259ae91904deb0d27c779314b36c: Status 404 returned error can't find the container with id 2ced6948a08d4385cf72af97da34d0b1b4af259ae91904deb0d27c779314b36c
Jul 13 18:51:44.389805 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-tls\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.389805 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.389805 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-grpc-tls\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.390029 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9hkx\" (UniqueName: \"kubernetes.io/projected/8867943a-2d0d-46ca-8551-61ccc60d5999-kube-api-access-m9hkx\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.390029 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.390029 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.390029 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.389991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8867943a-2d0d-46ca-8551-61ccc60d5999-metrics-client-ca\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.390242 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.390058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491288 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491288 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8867943a-2d0d-46ca-8551-61ccc60d5999-metrics-client-ca\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-tls\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491402    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-grpc-tls\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9hkx\" (UniqueName: \"kubernetes.io/projected/8867943a-2d0d-46ca-8551-61ccc60d5999-kube-api-access-m9hkx\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.491609 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.491473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.493444 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.493369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8867943a-2d0d-46ca-8551-61ccc60d5999-metrics-client-ca\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.494438 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.494408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.495479 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.495453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-tls\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.495926 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.495883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.496616 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.496596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-grpc-tls\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.497009 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.496893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.497336 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.497313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8867943a-2d0d-46ca-8551-61ccc60d5999-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.501623 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.501601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9hkx\" (UniqueName: \"kubernetes.io/projected/8867943a-2d0d-46ca-8551-61ccc60d5999-kube-api-access-m9hkx\") pod \"thanos-querier-77b594cd9d-2vcp6\" (UID: \"8867943a-2d0d-46ca-8551-61ccc60d5999\") " pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.612721 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.612690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:44.785473 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.785403    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"]
Jul 13 18:51:44.837412 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.837367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs" event={"ID":"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6","Type":"ContainerStarted","Data":"3559c930c307598c3ab7b1cd8b68001a3890741e85c8d086f4ce57df4bdbfc00"}
Jul 13 18:51:44.839683 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.839652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg" event={"ID":"9654d578-32ab-4a63-b69c-675cd9c1a38e","Type":"ContainerStarted","Data":"909d3e6df9911222a4c8946a7d41ab33adcf075d4628bb05743ff4d73b71ab2a"}
Jul 13 18:51:44.839815 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.839716    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg" event={"ID":"9654d578-32ab-4a63-b69c-675cd9c1a38e","Type":"ContainerStarted","Data":"f4ba9e278b5fb54ba2e1506e94ea949b54a512b0ef91bc225c666af9b919ae4d"}
Jul 13 18:51:44.839815 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.839748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg" event={"ID":"9654d578-32ab-4a63-b69c-675cd9c1a38e","Type":"ContainerStarted","Data":"a90bef9b7cad05f2bf3216d963d6c19bc16eb5ccffe910f58347685bd0ba1eed"}
Jul 13 18:51:44.841634 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.841599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-77cd8" event={"ID":"a79097ef-170e-4a96-9005-914625cb592b","Type":"ContainerStarted","Data":"0f3b17252e6da68d44d8c4bcdc31a8b9df3c5c1f8fa653708a83d6d24f484a31"}
Jul 13 18:51:44.841775 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.841757    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:51:44.843618 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.843590    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lvmn2" event={"ID":"27995e44-0dd7-4291-91d5-d0823bd49fb5","Type":"ContainerStarted","Data":"87d9663faae372379fe9b431ea7fe75736a86c50a4724bd8b5e6a3e3cb87a2cf"}
Jul 13 18:51:44.845148 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.845115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"2ced6948a08d4385cf72af97da34d0b1b4af259ae91904deb0d27c779314b36c"}
Jul 13 18:51:44.858517 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.858226    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-77cd8" podStartSLOduration=67.714153405 podStartE2EDuration="1m12.858208004s" totalImagesPullingTime="5.144054599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:44.857806851 +0000 UTC m=+73.905002007" watchObservedRunningTime="2026-07-13 18:51:44.858208004 +0000 UTC m=+73.905403161"
Jul 13 18:51:44.877998 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:44.877947    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lvmn2" podStartSLOduration=67.721264275 podStartE2EDuration="1m12.877928384s" totalImagesPullingTime="5.156664109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:44.877777174 +0000 UTC m=+73.924972330" watchObservedRunningTime="2026-07-13 18:51:44.877928384 +0000 UTC m=+73.925123540"
Jul 13 18:51:44.906630 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:44.906546    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8867943a_2d0d_46ca_8551_61ccc60d5999.slice/crio-b2e74abc980584b35ccd0e2430cdafe507c757efc4b356194f89c36529a321ea WatchSource:0}: Error finding container b2e74abc980584b35ccd0e2430cdafe507c757efc4b356194f89c36529a321ea: Status 404 returned error can't find the container with id b2e74abc980584b35ccd0e2430cdafe507c757efc4b356194f89c36529a321ea
Jul 13 18:51:45.760144 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:45.760114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h7v5n"
Jul 13 18:51:45.849859 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:45.849819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"b2e74abc980584b35ccd0e2430cdafe507c757efc4b356194f89c36529a321ea"}
Jul 13 18:51:46.618963 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.618923    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"]
Jul 13 18:51:46.622488 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.622465    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.627343 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.627286    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 13 18:51:46.627343 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.627310    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 13 18:51:46.627343 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.627333    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 18:51:46.627543 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.627372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qzmxb\""
Jul 13 18:51:46.627543 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.627286    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-36o60h4ckfp9f\""
Jul 13 18:51:46.627543 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.627310    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 13 18:51:46.653928 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.653902    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"]
Jul 13 18:51:46.717140 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-secret-metrics-server-tls\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.717306 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-client-ca-bundle\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.717306 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ba1367-adf0-4897-a4b5-80c22d4f60e9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.717306 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/45ba1367-adf0-4897-a4b5-80c22d4f60e9-audit-log\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.717306 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmxc\" (UniqueName: \"kubernetes.io/projected/45ba1367-adf0-4897-a4b5-80c22d4f60e9-kube-api-access-5hmxc\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.717478 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-secret-metrics-server-client-certs\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.717478 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.717363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/45ba1367-adf0-4897-a4b5-80c22d4f60e9-metrics-server-audit-profiles\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.818802 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.818760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ba1367-adf0-4897-a4b5-80c22d4f60e9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.818820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/45ba1367-adf0-4897-a4b5-80c22d4f60e9-audit-log\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.818951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmxc\" (UniqueName: \"kubernetes.io/projected/45ba1367-adf0-4897-a4b5-80c22d4f60e9-kube-api-access-5hmxc\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.819009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-secret-metrics-server-client-certs\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.819045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/45ba1367-adf0-4897-a4b5-80c22d4f60e9-metrics-server-audit-profiles\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.819127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-secret-metrics-server-tls\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.819172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-client-ca-bundle\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819275 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.819225    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/45ba1367-adf0-4897-a4b5-80c22d4f60e9-audit-log\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.819666 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.819632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45ba1367-adf0-4897-a4b5-80c22d4f60e9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.820109 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.820082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/45ba1367-adf0-4897-a4b5-80c22d4f60e9-metrics-server-audit-profiles\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.821819 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.821796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-client-ca-bundle\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.821937 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.821852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-secret-metrics-server-client-certs\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.821937 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.821896    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/45ba1367-adf0-4897-a4b5-80c22d4f60e9-secret-metrics-server-tls\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.830929 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.830908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmxc\" (UniqueName: \"kubernetes.io/projected/45ba1367-adf0-4897-a4b5-80c22d4f60e9-kube-api-access-5hmxc\") pod \"metrics-server-6c55bf4bb9-n69qb\" (UID: \"45ba1367-adf0-4897-a4b5-80c22d4f60e9\") " pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.856856 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.856819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg" event={"ID":"9654d578-32ab-4a63-b69c-675cd9c1a38e","Type":"ContainerStarted","Data":"b5a2dfa67248e4489f31c5feae5473588ac8b1a92eb7d24a7befaa5d5e9ac0c6"}
Jul 13 18:51:46.858720 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.858690    2581 generic.go:356] "Generic (PLEG): container finished" podID="af340ad3-9c0f-4ec5-8318-1cb719fa89ec" containerID="7729f89ba4298776f6d21a6b423b1f63caa8efb93687b3d5e5a557b31c5b9109" exitCode=0
Jul 13 18:51:46.858885 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.858846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerDied","Data":"7729f89ba4298776f6d21a6b423b1f63caa8efb93687b3d5e5a557b31c5b9109"}
Jul 13 18:51:46.863144 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.862983    2581 generic.go:356] "Generic (PLEG): container finished" podID="7da60cc6-7300-4da3-9c10-dba51771c6dd" containerID="254f70e18329bd9c5f1a11cd852de03d291d8cef6b35ff0262ae7b24ea268be4" exitCode=0
Jul 13 18:51:46.863144 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.863060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npsqx" event={"ID":"7da60cc6-7300-4da3-9c10-dba51771c6dd","Type":"ContainerDied","Data":"254f70e18329bd9c5f1a11cd852de03d291d8cef6b35ff0262ae7b24ea268be4"}
Jul 13 18:51:46.866435 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.866413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs" event={"ID":"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6","Type":"ContainerStarted","Data":"f568c443a2a7b8f59b649d59f50a7e248faaf4426a098a7c02006d1daa329bfe"}
Jul 13 18:51:46.866528 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.866455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs" event={"ID":"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6","Type":"ContainerStarted","Data":"e1ac24257746860b5c5efb518712bbd217f93aea4d491d522877f9de68e91c7d"}
Jul 13 18:51:46.866528 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.866474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs" event={"ID":"f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6","Type":"ContainerStarted","Data":"936b8564e2ac0ee5168285cb73626f1bac5bcffaaf66eb8408668d23de9b238c"}
Jul 13 18:51:46.879890 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.879840    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-csgmg" podStartSLOduration=3.111976337 podStartE2EDuration="4.879825448s" totalImagesPullingTime="1.767849111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:46.877484799 +0000 UTC m=+75.924679969" watchObservedRunningTime="2026-07-13 18:51:46.879825448 +0000 UTC m=+75.927020603"
Jul 13 18:51:46.898973 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.898948    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c586ccf97-q8h5b"]
Jul 13 18:51:46.902735 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.902712    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"]
Jul 13 18:51:46.907077 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.907053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:46.911618 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.911384    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 13 18:51:46.911892 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.911874    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-m9qzn\""
Jul 13 18:51:46.919832 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.919811    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"]
Jul 13 18:51:46.932874 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.932852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:51:46.939328 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:46.939283    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-nz8bs" podStartSLOduration=3.020295039 podStartE2EDuration="4.939268223s" totalImagesPullingTime="1.918973184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:46.938308648 +0000 UTC m=+75.985503801" watchObservedRunningTime="2026-07-13 18:51:46.939268223 +0000 UTC m=+75.986463378"
Jul 13 18:51:47.021073 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.021035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95b438c-9ea3-43ce-b2b8-3b7631a3731b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wwfb8\" (UID: \"f95b438c-9ea3-43ce-b2b8-3b7631a3731b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.021266 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.021081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw97m\" (UniqueName: \"kubernetes.io/projected/f95b438c-9ea3-43ce-b2b8-3b7631a3731b-kube-api-access-kw97m\") pod \"monitoring-plugin-5f68df7f58-wwfb8\" (UID: \"f95b438c-9ea3-43ce-b2b8-3b7631a3731b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.122714 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.122691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95b438c-9ea3-43ce-b2b8-3b7631a3731b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wwfb8\" (UID: \"f95b438c-9ea3-43ce-b2b8-3b7631a3731b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.122835 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.122722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw97m\" (UniqueName: \"kubernetes.io/projected/f95b438c-9ea3-43ce-b2b8-3b7631a3731b-kube-api-access-kw97m\") pod \"monitoring-plugin-5f68df7f58-wwfb8\" (UID: \"f95b438c-9ea3-43ce-b2b8-3b7631a3731b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.126044 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.125991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95b438c-9ea3-43ce-b2b8-3b7631a3731b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wwfb8\" (UID: \"f95b438c-9ea3-43ce-b2b8-3b7631a3731b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.137755 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.137693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw97m\" (UniqueName: \"kubernetes.io/projected/f95b438c-9ea3-43ce-b2b8-3b7631a3731b-kube-api-access-kw97m\") pod \"monitoring-plugin-5f68df7f58-wwfb8\" (UID: \"f95b438c-9ea3-43ce-b2b8-3b7631a3731b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.182766 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.182728    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"]
Jul 13 18:51:47.186819 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:47.186758    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ba1367_adf0_4897_a4b5_80c22d4f60e9.slice/crio-ddff733404bf9aa36455a21406fe1c2840c1f15e4854d5b93ad3cfdf085d599c WatchSource:0}: Error finding container ddff733404bf9aa36455a21406fe1c2840c1f15e4854d5b93ad3cfdf085d599c: Status 404 returned error can't find the container with id ddff733404bf9aa36455a21406fe1c2840c1f15e4854d5b93ad3cfdf085d599c
Jul 13 18:51:47.220955 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.220934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:47.362476 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.362446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"]
Jul 13 18:51:47.364318 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:47.364287    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95b438c_9ea3_43ce_b2b8_3b7631a3731b.slice/crio-b0944affc193166d1c5c4bd356791f41c81ba0800107ed8c7715b0d742abfc93 WatchSource:0}: Error finding container b0944affc193166d1c5c4bd356791f41c81ba0800107ed8c7715b0d742abfc93: Status 404 returned error can't find the container with id b0944affc193166d1c5c4bd356791f41c81ba0800107ed8c7715b0d742abfc93
Jul 13 18:51:47.872730 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.872687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8" event={"ID":"f95b438c-9ea3-43ce-b2b8-3b7631a3731b","Type":"ContainerStarted","Data":"b0944affc193166d1c5c4bd356791f41c81ba0800107ed8c7715b0d742abfc93"}
Jul 13 18:51:47.875674 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.875514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npsqx" event={"ID":"7da60cc6-7300-4da3-9c10-dba51771c6dd","Type":"ContainerStarted","Data":"c0a118fcc0f072b2e9ea1bf103948d904d0620b568ba85ef90ad971dd9ef3fc3"}
Jul 13 18:51:47.875674 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.875589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npsqx" event={"ID":"7da60cc6-7300-4da3-9c10-dba51771c6dd","Type":"ContainerStarted","Data":"8d918dff78fd02680357612a943b35749b0700954403e67ba449103762a46cdb"}
Jul 13 18:51:47.877239 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.877182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb" event={"ID":"45ba1367-adf0-4897-a4b5-80c22d4f60e9","Type":"ContainerStarted","Data":"ddff733404bf9aa36455a21406fe1c2840c1f15e4854d5b93ad3cfdf085d599c"}
Jul 13 18:51:47.880167 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.879867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"98682ac00adf44a3f4330f6b5c9afbb4f2003e5da6fbca48a782ef5c12836c27"}
Jul 13 18:51:47.880167 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.879933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"e682d2b338ccdf393ee4fb4d040440e6c07a4c7dab37b11a0f58369ba46a5697"}
Jul 13 18:51:47.880167 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.879974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"c07e6ec5ae905892689b86b4c1d7989184b5e44ccc2cf058d43fc5a1058da94f"}
Jul 13 18:51:47.910606 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:47.910534    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-npsqx" podStartSLOduration=3.737980173 podStartE2EDuration="5.910512868s" totalImagesPullingTime="2.086198413s" totalInitContainerRuntime="86.334282ms" isStatefulPod=true podCreationTimestamp="2026-07-13 18:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:47.909625177 +0000 UTC m=+76.956820335" watchObservedRunningTime="2026-07-13 18:51:47.910512868 +0000 UTC m=+76.957708025"
Jul 13 18:51:48.522535 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.522495    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 18:51:48.527526 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.527499    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.530761 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.530732    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.530992    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531389    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531691    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531724    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531691    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531826    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531837    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531905    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531928    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 18:51:48.532075 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.531961    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 18:51:48.532858 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.532211    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lh2vh\""
Jul 13 18:51:48.532858 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.532316    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 18:51:48.532858 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.532398    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 18:51:48.532858 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.532638    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b7uu75skpcd9\""
Jul 13 18:51:48.540181 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.540156    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 18:51:48.544648 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.544621    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 18:51:48.638479 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638641 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638641 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638748 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2x6r\" (UniqueName: \"kubernetes.io/projected/07268472-894b-4c51-b4d4-955217a17a41-kube-api-access-q2x6r\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638748 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.638865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-web-config\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639016 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07268472-894b-4c51-b4d4-955217a17a41-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639016 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639016 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07268472-894b-4c51-b4d4-955217a17a41-config-out\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639016 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.638977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639016 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639026    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-config\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.639250 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.639248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.733659 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.733617    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:51:48.740167 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740308 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-config\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740308 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740308 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740308 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740508 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740508 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2x6r\" (UniqueName: \"kubernetes.io/projected/07268472-894b-4c51-b4d4-955217a17a41-kube-api-access-q2x6r\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740508 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740508 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740508 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740508 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-web-config\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07268472-894b-4c51-b4d4-955217a17a41-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07268472-894b-4c51-b4d4-955217a17a41-config-out\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.740831 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.741249 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.740870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.741333 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.741305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.741401 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.741347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.742040 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.741891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.743174 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.742704    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.743174 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.742971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.744582 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.744156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.744982 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.744923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.746004 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.745967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.748841 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.748509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.748933 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.748847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07268472-894b-4c51-b4d4-955217a17a41-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.749668 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.749646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-config\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.749756 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.749728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.750603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.750171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.750603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.750214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.750603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.750375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07268472-894b-4c51-b4d4-955217a17a41-config-out\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.750603 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.750547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07268472-894b-4c51-b4d4-955217a17a41-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.750867 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.750841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.751672 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.751651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2x6r\" (UniqueName: \"kubernetes.io/projected/07268472-894b-4c51-b4d4-955217a17a41-kube-api-access-q2x6r\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.751800 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.751780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07268472-894b-4c51-b4d4-955217a17a41-web-config\") pod \"prometheus-k8s-0\" (UID: \"07268472-894b-4c51-b4d4-955217a17a41\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:48.842296 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:48.842206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:51:49.596349 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.596322    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 18:51:49.600928 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:51:49.600899    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07268472_894b_4c51_b4d4_955217a17a41.slice/crio-f8a6c3ab5ae5fe580ca332a662e290d130985024abfa028b2483c19f2ee055ee WatchSource:0}: Error finding container f8a6c3ab5ae5fe580ca332a662e290d130985024abfa028b2483c19f2ee055ee: Status 404 returned error can't find the container with id f8a6c3ab5ae5fe580ca332a662e290d130985024abfa028b2483c19f2ee055ee
Jul 13 18:51:49.891523 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.891495    2581 generic.go:356] "Generic (PLEG): container finished" podID="07268472-894b-4c51-b4d4-955217a17a41" containerID="b209fe953ae71c7d74ec10a86e638d3f533b0c64ee976afd1ebd67a433675232" exitCode=0
Jul 13 18:51:49.891659 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.891595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerDied","Data":"b209fe953ae71c7d74ec10a86e638d3f533b0c64ee976afd1ebd67a433675232"}
Jul 13 18:51:49.891726 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.891681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"f8a6c3ab5ae5fe580ca332a662e290d130985024abfa028b2483c19f2ee055ee"}
Jul 13 18:51:49.894918 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.894794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"04ea516a55c2eb9f2514bf9d0dd77e8feae5df3b45c6a3245e5574042cfaff44"}
Jul 13 18:51:49.894918 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.894891    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:49.895043 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.894936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"007faa2e27dd06066fa74af671c5bd2659c3e6fa68d6c6647464c2c9bf9925f2"}
Jul 13 18:51:49.895043 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.894975    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" event={"ID":"8867943a-2d0d-46ca-8551-61ccc60d5999","Type":"ContainerStarted","Data":"de0a63a064a0d70ed1f3bc909db3ec2531b16b008f8a68ba6b7888c399b31550"}
Jul 13 18:51:49.896762 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.896731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb" event={"ID":"45ba1367-adf0-4897-a4b5-80c22d4f60e9","Type":"ContainerStarted","Data":"97fa8c3e0e5ff47234b06188b516d007683361c11f47e89291e4249ec109d76f"}
Jul 13 18:51:49.900440 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.900330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"5fe774529bb41d35f18e06fde99461c1abe249b1e45d3fd88195f145e59491b3"}
Jul 13 18:51:49.900440 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.900403    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"3a2b8ec27a4c18bda4e0826f8fb0c93bc89cb4c19d8b43d962279dd7efe0f4bb"}
Jul 13 18:51:49.900649 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.900445    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"697e12968432f381649fa013b0813acde1782af89f7e8b2147b215be4a1bccaa"}
Jul 13 18:51:49.900649 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.900485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"c74eaf80715fbc5e6aa0635eb3e06832a877d60d1cf5a250eb0f5463121e8fed"}
Jul 13 18:51:49.900649 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.900523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"e01c9464116e50841f0b14ed089acb07b6d4ff6ce8269dc0513589f9a4500b84"}
Jul 13 18:51:49.902007 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.901986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8" event={"ID":"f95b438c-9ea3-43ce-b2b8-3b7631a3731b","Type":"ContainerStarted","Data":"65b34412872a85438159a1375bd5166d85bc237aa42cd517efbaf8020d1c88d3"}
Jul 13 18:51:49.902324 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.902276    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:49.907273 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.907255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8"
Jul 13 18:51:49.963179 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.963134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wwfb8" podStartSLOduration=1.906822424 podStartE2EDuration="3.963119406s" totalImagesPullingTime="2.056296982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:49.962131109 +0000 UTC m=+79.009326264" watchObservedRunningTime="2026-07-13 18:51:49.963119406 +0000 UTC m=+79.010314551"
Jul 13 18:51:49.998573 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:49.998510    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6" podStartSLOduration=1.602776644 podStartE2EDuration="5.998487816s" totalImagesPullingTime="4.395711172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:49.996657812 +0000 UTC m=+79.043852970" watchObservedRunningTime="2026-07-13 18:51:49.998487816 +0000 UTC m=+79.045682972"
Jul 13 18:51:50.034447 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:50.034341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb" podStartSLOduration=1.8013995029999998 podStartE2EDuration="4.034321628s" totalImagesPullingTime="2.232922125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:50.031660151 +0000 UTC m=+79.078855306" watchObservedRunningTime="2026-07-13 18:51:50.034321628 +0000 UTC m=+79.081516782"
Jul 13 18:51:50.910895 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:50.910834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af340ad3-9c0f-4ec5-8318-1cb719fa89ec","Type":"ContainerStarted","Data":"6ccabd2a1da5a6d9cfd504bdb0143e5633291a553c0d5ff71d8fd193d46451ca"}
Jul 13 18:51:50.941747 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:50.941696    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.862585909 podStartE2EDuration="7.941675119s" totalImagesPullingTime="4.056570422s" totalInitContainerRuntime="22.518788ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:50.938620131 +0000 UTC m=+79.985815285" watchObservedRunningTime="2026-07-13 18:51:50.941675119 +0000 UTC m=+79.988870287"
Jul 13 18:51:53.926885 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.926840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"6b7120ee5147636dc21ebae69bc03c7249828f303450fb4c3a351338ffd7aac1"}
Jul 13 18:51:53.927330 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.926919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"fa02aa6728e7eff4f5f6fbc5e5293cd665c3d89e3c4f85d9d18c1ca1dec775a4"}
Jul 13 18:51:53.927330 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.926946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"40d8983c7a466f0d14d071fb020d81ccc60cc743eccef9a4ed9359e5ed75db14"}
Jul 13 18:51:53.927330 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.926969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"7ca463abfe7808fdb52017120097114c857dc1c7159d1217488fd1c1a97009f1"}
Jul 13 18:51:53.927330 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.926999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"fe128df0daa7e711a79eee7d666946084d2753f8c78378b3a743df01dca20e4f"}
Jul 13 18:51:53.927330 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.927024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07268472-894b-4c51-b4d4-955217a17a41","Type":"ContainerStarted","Data":"a7b8b8041f13dbc7fd55316f74b465985e108bc05b3c29e705cf8c3d7cc85b5b"}
Jul 13 18:51:53.978102 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:53.978037    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.574383075 podStartE2EDuration="5.978015307s" totalImagesPullingTime="3.373649177s" totalInitContainerRuntime="29.983055ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:51:53.975848159 +0000 UTC m=+83.023043313" watchObservedRunningTime="2026-07-13 18:51:53.978015307 +0000 UTC m=+83.025210462"
Jul 13 18:51:55.917943 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:55.917913    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-77b594cd9d-2vcp6"
Jul 13 18:51:57.599143 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:57.599110    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-789c65455c-b5h9j"]
Jul 13 18:51:58.843017 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:51:58.842970    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:52:06.933228 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:06.933195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:52:06.933698 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:06.933250    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:52:11.925180 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:11.925105    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c586ccf97-q8h5b" podUID="a8c6589a-ade4-413c-828a-93d7ff21cf13" containerName="console" containerID="cri-o://bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840" gracePeriod=15
Jul 13 18:52:12.171172 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.171150    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:52:12.249458 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7hp9\" (UniqueName: \"kubernetes.io/projected/a8c6589a-ade4-413c-828a-93d7ff21cf13-kube-api-access-c7hp9\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249458 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249425    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-oauth-serving-cert\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249703 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249515    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-service-ca\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249703 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249596    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-oauth-config\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249703 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249630    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-trusted-ca-bundle\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-config\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249865 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-serving-cert\") pod \"a8c6589a-ade4-413c-828a-93d7ff21cf13\" (UID: \"a8c6589a-ade4-413c-828a-93d7ff21cf13\") "
Jul 13 18:52:12.249965 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249876    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:12.250020 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.249953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:12.250020 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.250002    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:12.250255 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.250226    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-config" (OuterVolumeSpecName: "console-config") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:12.250255 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.250252    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-service-ca\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:12.250379 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.250273    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-trusted-ca-bundle\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:12.250379 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.250324    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-oauth-serving-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:12.252177 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.252153    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:52:12.252272 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.252253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c6589a-ade4-413c-828a-93d7ff21cf13-kube-api-access-c7hp9" (OuterVolumeSpecName: "kube-api-access-c7hp9") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "kube-api-access-c7hp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:52:12.252314 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.252266    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8c6589a-ade4-413c-828a-93d7ff21cf13" (UID: "a8c6589a-ade4-413c-828a-93d7ff21cf13"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:52:12.351345 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.351296    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7hp9\" (UniqueName: \"kubernetes.io/projected/a8c6589a-ade4-413c-828a-93d7ff21cf13-kube-api-access-c7hp9\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:12.351345 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.351337    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-oauth-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:12.351345 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.351347    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:12.351345 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:12.351356    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8c6589a-ade4-413c-828a-93d7ff21cf13-console-serving-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:13.008726 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.008689    2581 generic.go:356] "Generic (PLEG): container finished" podID="a8c6589a-ade4-413c-828a-93d7ff21cf13" containerID="bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840" exitCode=0
Jul 13 18:52:13.008726 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.008728    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c586ccf97-q8h5b"
Jul 13 18:52:13.009196 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.008750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c586ccf97-q8h5b" event={"ID":"a8c6589a-ade4-413c-828a-93d7ff21cf13","Type":"ContainerDied","Data":"bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840"}
Jul 13 18:52:13.009196 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.008794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c586ccf97-q8h5b" event={"ID":"a8c6589a-ade4-413c-828a-93d7ff21cf13","Type":"ContainerDied","Data":"f26ee898202eb2e8639ac1e9c8418528abe117f5dabd32bafbd66c7354f2030f"}
Jul 13 18:52:13.009196 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.008816    2581 scope.go:117] "RemoveContainer" containerID="bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840"
Jul 13 18:52:13.017689 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.017671    2581 scope.go:117] "RemoveContainer" containerID="bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840"
Jul 13 18:52:13.017943 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:52:13.017924    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840\": container with ID starting with bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840 not found: ID does not exist" containerID="bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840"
Jul 13 18:52:13.017993 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.017952    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840"} err="failed to get container status \"bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840\": rpc error: code = NotFound desc = could not find container \"bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840\": container with ID starting with bf48d5a671c1e1654d7cca981db6f14cc2f3b9346d8a8996c258a6771b12e840 not found: ID does not exist"
Jul 13 18:52:13.029194 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.029170    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c586ccf97-q8h5b"]
Jul 13 18:52:13.030351 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.030334    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c586ccf97-q8h5b"]
Jul 13 18:52:13.506021 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:13.505987    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c6589a-ade4-413c-828a-93d7ff21cf13" path="/var/lib/kubelet/pods/a8c6589a-ade4-413c-828a-93d7ff21cf13/volumes"
Jul 13 18:52:15.852359 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:15.852327    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-77cd8"
Jul 13 18:52:22.619247 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:22.619165    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-789c65455c-b5h9j" podUID="2722a629-10f5-4ecb-9e4d-b7179d9a124f" containerName="console" containerID="cri-o://726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991" gracePeriod=15
Jul 13 18:52:22.948810 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:22.948783    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:52:23.052071 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052033    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-config\") pod \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") "
Jul 13 18:52:23.052242 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052088    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-serving-cert\") pod \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") "
Jul 13 18:52:23.052242 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052133    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfp24\" (UniqueName: \"kubernetes.io/projected/2722a629-10f5-4ecb-9e4d-b7179d9a124f-kube-api-access-kfp24\") pod \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") "
Jul 13 18:52:23.052242 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-oauth-config\") pod \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") "
Jul 13 18:52:23.052242 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052208    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-oauth-serving-cert\") pod \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") "
Jul 13 18:52:23.052457 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052256    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-service-ca\") pod \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\" (UID: \"2722a629-10f5-4ecb-9e4d-b7179d9a124f\") "
Jul 13 18:52:23.052613 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052461    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-config" (OuterVolumeSpecName: "console-config") pod "2722a629-10f5-4ecb-9e4d-b7179d9a124f" (UID: "2722a629-10f5-4ecb-9e4d-b7179d9a124f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:23.052781 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052752    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2722a629-10f5-4ecb-9e4d-b7179d9a124f" (UID: "2722a629-10f5-4ecb-9e4d-b7179d9a124f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:23.052857 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.052834    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-service-ca" (OuterVolumeSpecName: "service-ca") pod "2722a629-10f5-4ecb-9e4d-b7179d9a124f" (UID: "2722a629-10f5-4ecb-9e4d-b7179d9a124f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:52:23.053636 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.053601    2581 generic.go:356] "Generic (PLEG): container finished" podID="2722a629-10f5-4ecb-9e4d-b7179d9a124f" containerID="726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991" exitCode=0
Jul 13 18:52:23.053753 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.053662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-789c65455c-b5h9j"
Jul 13 18:52:23.053753 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.053666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-789c65455c-b5h9j" event={"ID":"2722a629-10f5-4ecb-9e4d-b7179d9a124f","Type":"ContainerDied","Data":"726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991"}
Jul 13 18:52:23.053866 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.053798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-789c65455c-b5h9j" event={"ID":"2722a629-10f5-4ecb-9e4d-b7179d9a124f","Type":"ContainerDied","Data":"35ea2a1cb7f42a7eab7e94d1177491cc26faba52309fc18068fec1d7e47008c6"}
Jul 13 18:52:23.053866 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.053824    2581 scope.go:117] "RemoveContainer" containerID="726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991"
Jul 13 18:52:23.054648 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.054622    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2722a629-10f5-4ecb-9e4d-b7179d9a124f" (UID: "2722a629-10f5-4ecb-9e4d-b7179d9a124f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:52:23.054781 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.054688    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2722a629-10f5-4ecb-9e4d-b7179d9a124f" (UID: "2722a629-10f5-4ecb-9e4d-b7179d9a124f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:52:23.054781 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.054686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2722a629-10f5-4ecb-9e4d-b7179d9a124f-kube-api-access-kfp24" (OuterVolumeSpecName: "kube-api-access-kfp24") pod "2722a629-10f5-4ecb-9e4d-b7179d9a124f" (UID: "2722a629-10f5-4ecb-9e4d-b7179d9a124f"). InnerVolumeSpecName "kube-api-access-kfp24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:52:23.068904 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.068888    2581 scope.go:117] "RemoveContainer" containerID="726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991"
Jul 13 18:52:23.069183 ip-10-0-134-48 kubenswrapper[2581]: E0713 18:52:23.069161    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991\": container with ID starting with 726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991 not found: ID does not exist" containerID="726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991"
Jul 13 18:52:23.069231 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.069192    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991"} err="failed to get container status \"726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991\": rpc error: code = NotFound desc = could not find container \"726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991\": container with ID starting with 726bca41d3b457e7e39222214f5a197a15510f647f4bbad229be25373abe0991 not found: ID does not exist"
Jul 13 18:52:23.153761 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.153729    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfp24\" (UniqueName: \"kubernetes.io/projected/2722a629-10f5-4ecb-9e4d-b7179d9a124f-kube-api-access-kfp24\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:23.153761 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.153757    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-oauth-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:23.153761 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.153766    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-oauth-serving-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:23.153943 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.153776    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-service-ca\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:23.153943 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.153784    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:23.153943 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.153792    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a629-10f5-4ecb-9e4d-b7179d9a124f-console-serving-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:52:23.376159 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.376128    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-789c65455c-b5h9j"]
Jul 13 18:52:23.378951 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.378929    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-789c65455c-b5h9j"]
Jul 13 18:52:23.506002 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:23.505968    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2722a629-10f5-4ecb-9e4d-b7179d9a124f" path="/var/lib/kubelet/pods/2722a629-10f5-4ecb-9e4d-b7179d9a124f/volumes"
Jul 13 18:52:26.939084 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:26.939047    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:52:26.942871 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:26.942846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c55bf4bb9-n69qb"
Jul 13 18:52:48.843173 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:48.843139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:52:48.862295 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:48.862267    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:52:49.177207 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:52:49.177181    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 18:55:31.341811 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:55:31.341780    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 18:55:31.343575 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:55:31.343540    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 18:55:31.346834 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:55:31.346816    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 18:58:31.985425 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985393    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-wr82s"]
Jul 13 18:58:31.985852 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985721    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c6589a-ade4-413c-828a-93d7ff21cf13" containerName="console"
Jul 13 18:58:31.985852 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985734    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c6589a-ade4-413c-828a-93d7ff21cf13" containerName="console"
Jul 13 18:58:31.985852 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985757    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2722a629-10f5-4ecb-9e4d-b7179d9a124f" containerName="console"
Jul 13 18:58:31.985852 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985762    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2722a629-10f5-4ecb-9e4d-b7179d9a124f" containerName="console"
Jul 13 18:58:31.985852 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985810    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2722a629-10f5-4ecb-9e4d-b7179d9a124f" containerName="console"
Jul 13 18:58:31.985852 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.985821    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c6589a-ade4-413c-828a-93d7ff21cf13" containerName="console"
Jul 13 18:58:31.987791 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.987771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wr82s"
Jul 13 18:58:31.990100 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.990074    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 13 18:58:31.990215 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.990077    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xps6j\""
Jul 13 18:58:31.990417 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.990401    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 13 18:58:31.990477 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.990422    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 13 18:58:31.996530 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:31.996509    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-wr82s"]
Jul 13 18:58:32.142223 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.142195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9ft\" (UniqueName: \"kubernetes.io/projected/b0a9be78-5dfc-4193-b031-b46af25fc6a6-kube-api-access-4t9ft\") pod \"s3-init-wr82s\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") " pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.142358 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.142298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9be78-5dfc-4193-b031-b46af25fc6a6-model-cache\") pod \"s3-init-wr82s\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") " pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.243139 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.243058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9ft\" (UniqueName: \"kubernetes.io/projected/b0a9be78-5dfc-4193-b031-b46af25fc6a6-kube-api-access-4t9ft\") pod \"s3-init-wr82s\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") " pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.243267 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.243162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9be78-5dfc-4193-b031-b46af25fc6a6-model-cache\") pod \"s3-init-wr82s\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") " pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.243542 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.243523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9be78-5dfc-4193-b031-b46af25fc6a6-model-cache\") pod \"s3-init-wr82s\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") " pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.251564 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.251533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9ft\" (UniqueName: \"kubernetes.io/projected/b0a9be78-5dfc-4193-b031-b46af25fc6a6-kube-api-access-4t9ft\") pod \"s3-init-wr82s\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") " pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.303668 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.303640    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wr82s"
Jul 13 18:58:32.428199 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.428111    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-wr82s"]
Jul 13 18:58:32.431030 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:58:32.430999    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0a9be78_5dfc_4193_b031_b46af25fc6a6.slice/crio-961463237dc062013490b788f4060df05122164f25c0d126b950ed21103ff1cd WatchSource:0}: Error finding container 961463237dc062013490b788f4060df05122164f25c0d126b950ed21103ff1cd: Status 404 returned error can't find the container with id 961463237dc062013490b788f4060df05122164f25c0d126b950ed21103ff1cd
Jul 13 18:58:32.435483 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.435467    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 18:58:32.462612 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:32.462578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wr82s" event={"ID":"b0a9be78-5dfc-4193-b031-b46af25fc6a6","Type":"ContainerStarted","Data":"961463237dc062013490b788f4060df05122164f25c0d126b950ed21103ff1cd"}
Jul 13 18:58:36.481866 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:36.481832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wr82s" event={"ID":"b0a9be78-5dfc-4193-b031-b46af25fc6a6","Type":"ContainerStarted","Data":"5023a1f851e2bc9ff94398474b55bdea545a7e20b1071c9bee2e1e05b82ce6c6"}
Jul 13 18:58:37.488153 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:37.488119    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerID="5023a1f851e2bc9ff94398474b55bdea545a7e20b1071c9bee2e1e05b82ce6c6" exitCode=0
Jul 13 18:58:37.488535 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:37.488202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wr82s" event={"ID":"b0a9be78-5dfc-4193-b031-b46af25fc6a6","Type":"ContainerDied","Data":"5023a1f851e2bc9ff94398474b55bdea545a7e20b1071c9bee2e1e05b82ce6c6"}
Jul 13 18:58:42.511190 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:42.511145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wr82s" event={"ID":"b0a9be78-5dfc-4193-b031-b46af25fc6a6","Type":"ContainerStarted","Data":"16f5a41db42b5d5548af211ec8e1c75439be1a908e0fce9c2200d9310e4d2f9a"}
Jul 13 18:58:42.531694 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:58:42.531637    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-wr82s" podStartSLOduration=2.389162464 podStartE2EDuration="11.531620868s" totalImagesPullingTime="8.06732986s" totalInitContainerRuntime="1.075128544s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:42.531222698 +0000 UTC m=+491.578417855" watchObservedRunningTime="2026-07-13 18:58:42.531620868 +0000 UTC m=+491.578816024"
Jul 13 18:59:47.762718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:47.762681    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerID="16f5a41db42b5d5548af211ec8e1c75439be1a908e0fce9c2200d9310e4d2f9a" exitCode=0
Jul 13 18:59:47.762718 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:47.762720    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wr82s" event={"ID":"b0a9be78-5dfc-4193-b031-b46af25fc6a6","Type":"ContainerDied","Data":"16f5a41db42b5d5548af211ec8e1c75439be1a908e0fce9c2200d9310e4d2f9a"}
Jul 13 18:59:48.893546 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:48.893523    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wr82s"
Jul 13 18:59:48.967105 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:48.967078    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t9ft\" (UniqueName: \"kubernetes.io/projected/b0a9be78-5dfc-4193-b031-b46af25fc6a6-kube-api-access-4t9ft\") pod \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") "
Jul 13 18:59:48.967268 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:48.967144    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9be78-5dfc-4193-b031-b46af25fc6a6-model-cache\") pod \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\" (UID: \"b0a9be78-5dfc-4193-b031-b46af25fc6a6\") "
Jul 13 18:59:48.969272 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:48.969229    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0a9be78-5dfc-4193-b031-b46af25fc6a6-kube-api-access-4t9ft" (OuterVolumeSpecName: "kube-api-access-4t9ft") pod "b0a9be78-5dfc-4193-b031-b46af25fc6a6" (UID: "b0a9be78-5dfc-4193-b031-b46af25fc6a6"). InnerVolumeSpecName "kube-api-access-4t9ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:59:48.986362 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:48.986330    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9be78-5dfc-4193-b031-b46af25fc6a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0a9be78-5dfc-4193-b031-b46af25fc6a6" (UID: "b0a9be78-5dfc-4193-b031-b46af25fc6a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:59:49.067761 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:49.067687    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9be78-5dfc-4193-b031-b46af25fc6a6-model-cache\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:59:49.067761 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:49.067714    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t9ft\" (UniqueName: \"kubernetes.io/projected/b0a9be78-5dfc-4193-b031-b46af25fc6a6-kube-api-access-4t9ft\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 18:59:49.771671 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:49.771596    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wr82s"
Jul 13 18:59:49.771671 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:49.771623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wr82s" event={"ID":"b0a9be78-5dfc-4193-b031-b46af25fc6a6","Type":"ContainerDied","Data":"961463237dc062013490b788f4060df05122164f25c0d126b950ed21103ff1cd"}
Jul 13 18:59:49.771671 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:49.771656    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="961463237dc062013490b788f4060df05122164f25c0d126b950ed21103ff1cd"
Jul 13 18:59:50.520542 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.520507    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"]
Jul 13 18:59:50.520920 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.520849    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerName="s3-init"
Jul 13 18:59:50.520920 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.520860    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerName="s3-init"
Jul 13 18:59:50.520920 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.520874    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerName="download-hf-model"
Jul 13 18:59:50.520920 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.520880    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerName="download-hf-model"
Jul 13 18:59:50.521048 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.520939    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0a9be78-5dfc-4193-b031-b46af25fc6a6" containerName="s3-init"
Jul 13 18:59:50.522795 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.522777    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.525153 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.525130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 13 18:59:50.525365 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.525347    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xps6j\""
Jul 13 18:59:50.525531 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.525513    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 13 18:59:50.526051 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.526037    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 13 18:59:50.532323 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.532302    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"]
Jul 13 18:59:50.576190 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.576164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c4bc81c1-3712-4f89-8153-8514d6c12d7c-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-2wkxh\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.576353 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.576200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zvck\" (UniqueName: \"kubernetes.io/projected/c4bc81c1-3712-4f89-8153-8514d6c12d7c-kube-api-access-7zvck\") pod \"seaweedfs-tls-custom-6644bf5d6c-2wkxh\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.677482 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.677453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c4bc81c1-3712-4f89-8153-8514d6c12d7c-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-2wkxh\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.677482 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.677490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zvck\" (UniqueName: \"kubernetes.io/projected/c4bc81c1-3712-4f89-8153-8514d6c12d7c-kube-api-access-7zvck\") pod \"seaweedfs-tls-custom-6644bf5d6c-2wkxh\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.677835 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.677817    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c4bc81c1-3712-4f89-8153-8514d6c12d7c-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-2wkxh\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.685642 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.685620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zvck\" (UniqueName: \"kubernetes.io/projected/c4bc81c1-3712-4f89-8153-8514d6c12d7c-kube-api-access-7zvck\") pod \"seaweedfs-tls-custom-6644bf5d6c-2wkxh\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.832637 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.832508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 18:59:50.969074 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:50.969052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"]
Jul 13 18:59:50.971689 ip-10-0-134-48 kubenswrapper[2581]: W0713 18:59:50.971661    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bc81c1_3712_4f89_8153_8514d6c12d7c.slice/crio-0dab41a89729cf8f54a78cd5ea094ff82d18e82354fb3fd8e330b38833e50550 WatchSource:0}: Error finding container 0dab41a89729cf8f54a78cd5ea094ff82d18e82354fb3fd8e330b38833e50550: Status 404 returned error can't find the container with id 0dab41a89729cf8f54a78cd5ea094ff82d18e82354fb3fd8e330b38833e50550
Jul 13 18:59:51.782436 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:51.782391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh" event={"ID":"c4bc81c1-3712-4f89-8153-8514d6c12d7c","Type":"ContainerStarted","Data":"0dab41a89729cf8f54a78cd5ea094ff82d18e82354fb3fd8e330b38833e50550"}
Jul 13 18:59:54.797797 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:54.797762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh" event={"ID":"c4bc81c1-3712-4f89-8153-8514d6c12d7c","Type":"ContainerStarted","Data":"f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4"}
Jul 13 18:59:54.815251 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:54.815204    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh" podStartSLOduration=1.918565334 podStartE2EDuration="4.815189193s" totalImagesPullingTime="2.896623859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:54.813822591 +0000 UTC m=+563.861017737" watchObservedRunningTime="2026-07-13 18:59:54.815189193 +0000 UTC m=+563.862384348"
Jul 13 18:59:55.808495 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:55.808464    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"]
Jul 13 18:59:56.805767 ip-10-0-134-48 kubenswrapper[2581]: I0713 18:59:56.805729    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh" podUID="c4bc81c1-3712-4f89-8153-8514d6c12d7c" containerName="seaweedfs-tls-custom" containerID="cri-o://f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4" gracePeriod=30
Jul 13 19:00:24.537953 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.537927    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 19:00:24.672645 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.672615    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zvck\" (UniqueName: \"kubernetes.io/projected/c4bc81c1-3712-4f89-8153-8514d6c12d7c-kube-api-access-7zvck\") pod \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") "
Jul 13 19:00:24.672825 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.672715    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c4bc81c1-3712-4f89-8153-8514d6c12d7c-data\") pod \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\" (UID: \"c4bc81c1-3712-4f89-8153-8514d6c12d7c\") "
Jul 13 19:00:24.674015 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.673988    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bc81c1-3712-4f89-8153-8514d6c12d7c-data" (OuterVolumeSpecName: "data") pod "c4bc81c1-3712-4f89-8153-8514d6c12d7c" (UID: "c4bc81c1-3712-4f89-8153-8514d6c12d7c"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:00:24.674703 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.674681    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bc81c1-3712-4f89-8153-8514d6c12d7c-kube-api-access-7zvck" (OuterVolumeSpecName: "kube-api-access-7zvck") pod "c4bc81c1-3712-4f89-8153-8514d6c12d7c" (UID: "c4bc81c1-3712-4f89-8153-8514d6c12d7c"). InnerVolumeSpecName "kube-api-access-7zvck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:00:24.774016 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.773978    2581 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c4bc81c1-3712-4f89-8153-8514d6c12d7c-data\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:00:24.774016 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.774008    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zvck\" (UniqueName: \"kubernetes.io/projected/c4bc81c1-3712-4f89-8153-8514d6c12d7c-kube-api-access-7zvck\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:00:24.920709 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.920672    2581 generic.go:356] "Generic (PLEG): container finished" podID="c4bc81c1-3712-4f89-8153-8514d6c12d7c" containerID="f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4" exitCode=0
Jul 13 19:00:24.920897 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.920724    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"
Jul 13 19:00:24.920897 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.920728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh" event={"ID":"c4bc81c1-3712-4f89-8153-8514d6c12d7c","Type":"ContainerDied","Data":"f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4"}
Jul 13 19:00:24.920897 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.920808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh" event={"ID":"c4bc81c1-3712-4f89-8153-8514d6c12d7c","Type":"ContainerDied","Data":"0dab41a89729cf8f54a78cd5ea094ff82d18e82354fb3fd8e330b38833e50550"}
Jul 13 19:00:24.920897 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.920835    2581 scope.go:117] "RemoveContainer" containerID="f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4"
Jul 13 19:00:24.930513 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.930496    2581 scope.go:117] "RemoveContainer" containerID="f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4"
Jul 13 19:00:24.930747 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:00:24.930727    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4\": container with ID starting with f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4 not found: ID does not exist" containerID="f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4"
Jul 13 19:00:24.930801 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.930754    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4"} err="failed to get container status \"f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4\": rpc error: code = NotFound desc = could not find container \"f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4\": container with ID starting with f71f69533430a776c27f6f260cb4aa32d3352208d4244569e5543f10078716e4 not found: ID does not exist"
Jul 13 19:00:24.943368 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.943345    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"]
Jul 13 19:00:24.948367 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.948348    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-2wkxh"]
Jul 13 19:00:24.992173 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.992143    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"]
Jul 13 19:00:24.992453 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.992442    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4bc81c1-3712-4f89-8153-8514d6c12d7c" containerName="seaweedfs-tls-custom"
Jul 13 19:00:24.992500 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.992454    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4bc81c1-3712-4f89-8153-8514d6c12d7c" containerName="seaweedfs-tls-custom"
Jul 13 19:00:24.992532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.992515    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4bc81c1-3712-4f89-8153-8514d6c12d7c" containerName="seaweedfs-tls-custom"
Jul 13 19:00:24.995176 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.995161    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:24.997200 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.997178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\""
Jul 13 19:00:24.997311 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.997277    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 13 19:00:24.997405 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.997391    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 13 19:00:24.997491 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.997477    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 13 19:00:24.997713 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:24.997700    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xps6j\""
Jul 13 19:00:25.008109 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.008088    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"]
Jul 13 19:00:25.076436 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.076396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/13fdfea4-f476-45d5-aa29-5b935bd5f196-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.076634 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.076498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/13fdfea4-f476-45d5-aa29-5b935bd5f196-data\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.076634 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.076579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thm6t\" (UniqueName: \"kubernetes.io/projected/13fdfea4-f476-45d5-aa29-5b935bd5f196-kube-api-access-thm6t\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.177837 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.177796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/13fdfea4-f476-45d5-aa29-5b935bd5f196-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.178017 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.177880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/13fdfea4-f476-45d5-aa29-5b935bd5f196-data\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.178017 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.177914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thm6t\" (UniqueName: \"kubernetes.io/projected/13fdfea4-f476-45d5-aa29-5b935bd5f196-kube-api-access-thm6t\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.178354 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.178303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/13fdfea4-f476-45d5-aa29-5b935bd5f196-data\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.180205 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.180182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/13fdfea4-f476-45d5-aa29-5b935bd5f196-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.187669 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.187650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thm6t\" (UniqueName: \"kubernetes.io/projected/13fdfea4-f476-45d5-aa29-5b935bd5f196-kube-api-access-thm6t\") pod \"seaweedfs-tls-custom-8b44d767d-qkf7z\" (UID: \"13fdfea4-f476-45d5-aa29-5b935bd5f196\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.304137 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.304102    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"
Jul 13 19:00:25.431423 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.431342    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-qkf7z"]
Jul 13 19:00:25.434169 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:00:25.434145    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fdfea4_f476_45d5_aa29_5b935bd5f196.slice/crio-a23858f7ce7f2d2568cbd301a58c24885641e6a36216866089ecad0fcc5626d4 WatchSource:0}: Error finding container a23858f7ce7f2d2568cbd301a58c24885641e6a36216866089ecad0fcc5626d4: Status 404 returned error can't find the container with id a23858f7ce7f2d2568cbd301a58c24885641e6a36216866089ecad0fcc5626d4
Jul 13 19:00:25.506605 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.506578    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4bc81c1-3712-4f89-8153-8514d6c12d7c" path="/var/lib/kubelet/pods/c4bc81c1-3712-4f89-8153-8514d6c12d7c/volumes"
Jul 13 19:00:25.927608 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.927576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z" event={"ID":"13fdfea4-f476-45d5-aa29-5b935bd5f196","Type":"ContainerStarted","Data":"85bb856ffb11c2e754ce4b2c857ab0ac7f65d2921770f11c1a7af4185f67149d"}
Jul 13 19:00:25.927963 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.927622    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z" event={"ID":"13fdfea4-f476-45d5-aa29-5b935bd5f196","Type":"ContainerStarted","Data":"a23858f7ce7f2d2568cbd301a58c24885641e6a36216866089ecad0fcc5626d4"}
Jul 13 19:00:25.946827 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:25.946780    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-qkf7z" podStartSLOduration=1.591992372 podStartE2EDuration="1.946765635s" totalImagesPullingTime="354.773263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:25.945097115 +0000 UTC m=+594.992292269" watchObservedRunningTime="2026-07-13 19:00:25.946765635 +0000 UTC m=+594.993960789"
Jul 13 19:00:26.248450 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.248411    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-tggj9"]
Jul 13 19:00:26.250720 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.250698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.259512 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.259487    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-tggj9"]
Jul 13 19:00:26.390754 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.390720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hmvm\" (UniqueName: \"kubernetes.io/projected/8929db73-95e4-4b09-b70f-52a6929c51c9-kube-api-access-9hmvm\") pod \"s3-tls-init-custom-tggj9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") " pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.390928 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.390826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8929db73-95e4-4b09-b70f-52a6929c51c9-model-cache\") pod \"s3-tls-init-custom-tggj9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") " pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.491401 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.491356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8929db73-95e4-4b09-b70f-52a6929c51c9-model-cache\") pod \"s3-tls-init-custom-tggj9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") " pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.491535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.491446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hmvm\" (UniqueName: \"kubernetes.io/projected/8929db73-95e4-4b09-b70f-52a6929c51c9-kube-api-access-9hmvm\") pod \"s3-tls-init-custom-tggj9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") " pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.491774 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.491753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8929db73-95e4-4b09-b70f-52a6929c51c9-model-cache\") pod \"s3-tls-init-custom-tggj9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") " pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.505980 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.505917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hmvm\" (UniqueName: \"kubernetes.io/projected/8929db73-95e4-4b09-b70f-52a6929c51c9-kube-api-access-9hmvm\") pod \"s3-tls-init-custom-tggj9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") " pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.567622 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.567578    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:26.694293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.694261    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-tggj9"]
Jul 13 19:00:26.697868 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:00:26.697844    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8929db73_95e4_4b09_b70f_52a6929c51c9.slice/crio-8e12959db4022df8cb6d0c6fe83945678af952fc1197bac1a1379a250914f095 WatchSource:0}: Error finding container 8e12959db4022df8cb6d0c6fe83945678af952fc1197bac1a1379a250914f095: Status 404 returned error can't find the container with id 8e12959db4022df8cb6d0c6fe83945678af952fc1197bac1a1379a250914f095
Jul 13 19:00:26.933455 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.933415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tggj9" event={"ID":"8929db73-95e4-4b09-b70f-52a6929c51c9","Type":"ContainerStarted","Data":"449b716d84a1cdd551c62e4d647b05aefaa29e56c7332dbda398bebd53f6d598"}
Jul 13 19:00:26.933455 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:26.933460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tggj9" event={"ID":"8929db73-95e4-4b09-b70f-52a6929c51c9","Type":"ContainerStarted","Data":"8e12959db4022df8cb6d0c6fe83945678af952fc1197bac1a1379a250914f095"}
Jul 13 19:00:29.952631 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:29.952591    2581 generic.go:356] "Generic (PLEG): container finished" podID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerID="449b716d84a1cdd551c62e4d647b05aefaa29e56c7332dbda398bebd53f6d598" exitCode=0
Jul 13 19:00:29.952988 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:29.952664    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tggj9" event={"ID":"8929db73-95e4-4b09-b70f-52a6929c51c9","Type":"ContainerDied","Data":"449b716d84a1cdd551c62e4d647b05aefaa29e56c7332dbda398bebd53f6d598"}
Jul 13 19:00:30.958534 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:30.958502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tggj9" event={"ID":"8929db73-95e4-4b09-b70f-52a6929c51c9","Type":"ContainerStarted","Data":"e230b2673b79941504e68ce6f111299b08666c7ff27b3b8c96a694ef112d1de8"}
Jul 13 19:00:30.976831 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:30.976782    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-tggj9" podStartSLOduration=2.766405437 podStartE2EDuration="4.97676505s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.210359613s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:30.975494141 +0000 UTC m=+600.022689299" watchObservedRunningTime="2026-07-13 19:00:30.97676505 +0000 UTC m=+600.023960205"
Jul 13 19:00:31.370940 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:31.370868    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:00:31.371137 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:31.371121    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:00:36.987350 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:36.987316    2581 generic.go:356] "Generic (PLEG): container finished" podID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerID="e230b2673b79941504e68ce6f111299b08666c7ff27b3b8c96a694ef112d1de8" exitCode=0
Jul 13 19:00:36.987792 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:36.987389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tggj9" event={"ID":"8929db73-95e4-4b09-b70f-52a6929c51c9","Type":"ContainerDied","Data":"e230b2673b79941504e68ce6f111299b08666c7ff27b3b8c96a694ef112d1de8"}
Jul 13 19:00:38.141506 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.141485    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:38.195133 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.195106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hmvm\" (UniqueName: \"kubernetes.io/projected/8929db73-95e4-4b09-b70f-52a6929c51c9-kube-api-access-9hmvm\") pod \"8929db73-95e4-4b09-b70f-52a6929c51c9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") "
Jul 13 19:00:38.195263 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.195152    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8929db73-95e4-4b09-b70f-52a6929c51c9-model-cache\") pod \"8929db73-95e4-4b09-b70f-52a6929c51c9\" (UID: \"8929db73-95e4-4b09-b70f-52a6929c51c9\") "
Jul 13 19:00:38.197113 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.197087    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8929db73-95e4-4b09-b70f-52a6929c51c9-kube-api-access-9hmvm" (OuterVolumeSpecName: "kube-api-access-9hmvm") pod "8929db73-95e4-4b09-b70f-52a6929c51c9" (UID: "8929db73-95e4-4b09-b70f-52a6929c51c9"). InnerVolumeSpecName "kube-api-access-9hmvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:00:38.217425 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.217400    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8929db73-95e4-4b09-b70f-52a6929c51c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "8929db73-95e4-4b09-b70f-52a6929c51c9" (UID: "8929db73-95e4-4b09-b70f-52a6929c51c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:00:38.295827 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.295771    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hmvm\" (UniqueName: \"kubernetes.io/projected/8929db73-95e4-4b09-b70f-52a6929c51c9-kube-api-access-9hmvm\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:00:38.295827 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.295792    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8929db73-95e4-4b09-b70f-52a6929c51c9-model-cache\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:00:38.996777 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.996741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tggj9" event={"ID":"8929db73-95e4-4b09-b70f-52a6929c51c9","Type":"ContainerDied","Data":"8e12959db4022df8cb6d0c6fe83945678af952fc1197bac1a1379a250914f095"}
Jul 13 19:00:38.996777 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.996782    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tggj9"
Jul 13 19:00:38.996968 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:38.996784    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e12959db4022df8cb6d0c6fe83945678af952fc1197bac1a1379a250914f095"
Jul 13 19:00:39.609726 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.609692    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"]
Jul 13 19:00:39.610077 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.609993    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerName="s3-tls-init-custom"
Jul 13 19:00:39.610077 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.610003    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerName="s3-tls-init-custom"
Jul 13 19:00:39.610077 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.610027    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerName="download-hf-model"
Jul 13 19:00:39.610077 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.610032    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerName="download-hf-model"
Jul 13 19:00:39.610077 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.610077    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8929db73-95e4-4b09-b70f-52a6929c51c9" containerName="s3-tls-init-custom"
Jul 13 19:00:39.613332 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.613316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.615590 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.615538    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving\""
Jul 13 19:00:39.615712 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.615647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Jul 13 19:00:39.622593 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.622541    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"]
Jul 13 19:00:39.707768 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.707739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/432accba-c370-4e93-baea-274a9cc55963-data\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.707937 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.707778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.707937 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.707805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh8vh\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-kube-api-access-zh8vh\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.808181 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.808146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/432accba-c370-4e93-baea-274a9cc55963-data\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.808363 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.808198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.808363 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.808236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh8vh\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-kube-api-access-zh8vh\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.808471 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:00:39.808361    2581 projected.go:266] Couldn't get secret kserve/seaweedfs-tls-serving: secret "seaweedfs-tls-serving" not found
Jul 13 19:00:39.808471 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:00:39.808383    2581 projected.go:196] Error preparing data for projected volume seaweedfs-tls-serving for pod kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp: secret "seaweedfs-tls-serving" not found
Jul 13 19:00:39.808471 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:00:39.808445    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-seaweedfs-tls-serving podName:432accba-c370-4e93-baea-274a9cc55963 nodeName:}" failed. No retries permitted until 2026-07-13 19:00:40.308427592 +0000 UTC m=+609.355622729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "seaweedfs-tls-serving" (UniqueName: "kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-seaweedfs-tls-serving") pod "seaweedfs-tls-serving-67b7477c6d-6t5vp" (UID: "432accba-c370-4e93-baea-274a9cc55963") : secret "seaweedfs-tls-serving" not found
Jul 13 19:00:39.808471 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.808464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/432accba-c370-4e93-baea-274a9cc55963-data\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:39.818773 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:39.818753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh8vh\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-kube-api-access-zh8vh\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:40.313719 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:40.313685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:40.316006 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:40.315988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/432accba-c370-4e93-baea-274a9cc55963-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-6t5vp\" (UID: \"432accba-c370-4e93-baea-274a9cc55963\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:40.522343 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:40.522310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"
Jul 13 19:00:40.646762 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:40.646726    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp"]
Jul 13 19:00:40.649349 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:00:40.649308    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432accba_c370_4e93_baea_274a9cc55963.slice/crio-15f729a26641372983f9de0a2891bf40e6a26fc6474493dc038761503f0ed139 WatchSource:0}: Error finding container 15f729a26641372983f9de0a2891bf40e6a26fc6474493dc038761503f0ed139: Status 404 returned error can't find the container with id 15f729a26641372983f9de0a2891bf40e6a26fc6474493dc038761503f0ed139
Jul 13 19:00:41.006105 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:41.006071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp" event={"ID":"432accba-c370-4e93-baea-274a9cc55963","Type":"ContainerStarted","Data":"15f729a26641372983f9de0a2891bf40e6a26fc6474493dc038761503f0ed139"}
Jul 13 19:00:42.011845 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.011808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp" event={"ID":"432accba-c370-4e93-baea-274a9cc55963","Type":"ContainerStarted","Data":"05f0049af0d333f09f5b5645a6bcd2a97636088540500bd1b1ab9a4485e12f23"}
Jul 13 19:00:42.028884 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.028825    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-serving-67b7477c6d-6t5vp" podStartSLOduration=2.557630385 podStartE2EDuration="3.028804964s" totalImagesPullingTime="471.174579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:42.028133284 +0000 UTC m=+611.075328439" watchObservedRunningTime="2026-07-13 19:00:42.028804964 +0000 UTC m=+611.076000120"
Jul 13 19:00:42.566794 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.566764    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-zq9zh"]
Jul 13 19:00:42.571487 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.571470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.579288 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.579268    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-zq9zh"]
Jul 13 19:00:42.634808 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.634782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffba2a3f-aaab-4f99-97ad-d358e9089955-model-cache\") pod \"s3-tls-init-serving-zq9zh\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") " pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.634913 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.634837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxzk2\" (UniqueName: \"kubernetes.io/projected/ffba2a3f-aaab-4f99-97ad-d358e9089955-kube-api-access-wxzk2\") pod \"s3-tls-init-serving-zq9zh\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") " pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.735840 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.735809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffba2a3f-aaab-4f99-97ad-d358e9089955-model-cache\") pod \"s3-tls-init-serving-zq9zh\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") " pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.735970 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.735861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxzk2\" (UniqueName: \"kubernetes.io/projected/ffba2a3f-aaab-4f99-97ad-d358e9089955-kube-api-access-wxzk2\") pod \"s3-tls-init-serving-zq9zh\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") " pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.736177 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.736160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffba2a3f-aaab-4f99-97ad-d358e9089955-model-cache\") pod \"s3-tls-init-serving-zq9zh\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") " pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.744415 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.744384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxzk2\" (UniqueName: \"kubernetes.io/projected/ffba2a3f-aaab-4f99-97ad-d358e9089955-kube-api-access-wxzk2\") pod \"s3-tls-init-serving-zq9zh\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") " pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:42.894948 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:42.894925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:43.019056 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:43.019003    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-zq9zh"]
Jul 13 19:00:43.021123 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:00:43.021094    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffba2a3f_aaab_4f99_97ad_d358e9089955.slice/crio-b11d8e1c12377c5dcbe98b3eaf440a179e2f46bfd0f62433056097c29936f101 WatchSource:0}: Error finding container b11d8e1c12377c5dcbe98b3eaf440a179e2f46bfd0f62433056097c29936f101: Status 404 returned error can't find the container with id b11d8e1c12377c5dcbe98b3eaf440a179e2f46bfd0f62433056097c29936f101
Jul 13 19:00:44.025216 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:44.025172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-zq9zh" event={"ID":"ffba2a3f-aaab-4f99-97ad-d358e9089955","Type":"ContainerStarted","Data":"5c5836286d5b18e5f8a8622a676bb604f367ffd662280a0d8ba86744315abd4f"}
Jul 13 19:00:44.025216 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:44.025223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-zq9zh" event={"ID":"ffba2a3f-aaab-4f99-97ad-d358e9089955","Type":"ContainerStarted","Data":"b11d8e1c12377c5dcbe98b3eaf440a179e2f46bfd0f62433056097c29936f101"}
Jul 13 19:00:45.030117 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:45.030038    2581 generic.go:356] "Generic (PLEG): container finished" podID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerID="5c5836286d5b18e5f8a8622a676bb604f367ffd662280a0d8ba86744315abd4f" exitCode=0
Jul 13 19:00:45.030465 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:45.030122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-zq9zh" event={"ID":"ffba2a3f-aaab-4f99-97ad-d358e9089955","Type":"ContainerDied","Data":"5c5836286d5b18e5f8a8622a676bb604f367ffd662280a0d8ba86744315abd4f"}
Jul 13 19:00:46.036424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:46.036384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-zq9zh" event={"ID":"ffba2a3f-aaab-4f99-97ad-d358e9089955","Type":"ContainerStarted","Data":"7b6d7905b98f412b841c2770f4f2e6e322a1f0e81adf06c8162c9920d82ed1eb"}
Jul 13 19:00:46.053483 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:46.053430    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-zq9zh" podStartSLOduration=2.468088384 podStartE2EDuration="4.053411363s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.585322979s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:46.052704587 +0000 UTC m=+615.099899736" watchObservedRunningTime="2026-07-13 19:00:46.053411363 +0000 UTC m=+615.100606519"
Jul 13 19:00:51.062250 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:51.062162    2581 generic.go:356] "Generic (PLEG): container finished" podID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerID="7b6d7905b98f412b841c2770f4f2e6e322a1f0e81adf06c8162c9920d82ed1eb" exitCode=0
Jul 13 19:00:51.062250 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:51.062230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-zq9zh" event={"ID":"ffba2a3f-aaab-4f99-97ad-d358e9089955","Type":"ContainerDied","Data":"7b6d7905b98f412b841c2770f4f2e6e322a1f0e81adf06c8162c9920d82ed1eb"}
Jul 13 19:00:52.228355 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.228328    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:00:52.325224 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.325087    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxzk2\" (UniqueName: \"kubernetes.io/projected/ffba2a3f-aaab-4f99-97ad-d358e9089955-kube-api-access-wxzk2\") pod \"ffba2a3f-aaab-4f99-97ad-d358e9089955\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") "
Jul 13 19:00:52.325365 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.325279    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffba2a3f-aaab-4f99-97ad-d358e9089955-model-cache\") pod \"ffba2a3f-aaab-4f99-97ad-d358e9089955\" (UID: \"ffba2a3f-aaab-4f99-97ad-d358e9089955\") "
Jul 13 19:00:52.327117 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.327093    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffba2a3f-aaab-4f99-97ad-d358e9089955-kube-api-access-wxzk2" (OuterVolumeSpecName: "kube-api-access-wxzk2") pod "ffba2a3f-aaab-4f99-97ad-d358e9089955" (UID: "ffba2a3f-aaab-4f99-97ad-d358e9089955"). InnerVolumeSpecName "kube-api-access-wxzk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:00:52.349231 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.349203    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffba2a3f-aaab-4f99-97ad-d358e9089955-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffba2a3f-aaab-4f99-97ad-d358e9089955" (UID: "ffba2a3f-aaab-4f99-97ad-d358e9089955"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:00:52.426121 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.426090    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffba2a3f-aaab-4f99-97ad-d358e9089955-model-cache\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:00:52.426121 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:52.426117    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxzk2\" (UniqueName: \"kubernetes.io/projected/ffba2a3f-aaab-4f99-97ad-d358e9089955-kube-api-access-wxzk2\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:00:53.071970 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:53.071889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-zq9zh" event={"ID":"ffba2a3f-aaab-4f99-97ad-d358e9089955","Type":"ContainerDied","Data":"b11d8e1c12377c5dcbe98b3eaf440a179e2f46bfd0f62433056097c29936f101"}
Jul 13 19:00:53.071970 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:53.071934    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b11d8e1c12377c5dcbe98b3eaf440a179e2f46bfd0f62433056097c29936f101"
Jul 13 19:00:53.071970 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:00:53.071907    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-zq9zh"
Jul 13 19:01:05.991711 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.991678    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"]
Jul 13 19:01:05.992118 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.991985    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerName="download-hf-model"
Jul 13 19:01:05.992118 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.991996    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerName="download-hf-model"
Jul 13 19:01:05.992118 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.992015    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerName="s3-tls-init-serving"
Jul 13 19:01:05.992118 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.992022    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerName="s3-tls-init-serving"
Jul 13 19:01:05.992118 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.992081    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffba2a3f-aaab-4f99-97ad-d358e9089955" containerName="s3-tls-init-serving"
Jul 13 19:01:05.994688 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.994670    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:05.996894 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.996874    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 13 19:01:05.997078 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.997065    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 13 19:01:05.997305 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.997285    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 13 19:01:05.997416 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.997365    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 13 19:01:05.997416 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.997365    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 13 19:01:05.997416 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.997396    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 13 19:01:05.997585 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:05.997419    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-q9f8j\""
Jul 13 19:01:06.010600 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.010580    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"]
Jul 13 19:01:06.146615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.146580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/88647fa7-2e0a-4fa8-b77b-3f449a52d644-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.146778 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.146642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.146778 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.146681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f268\" (UniqueName: \"kubernetes.io/projected/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-api-access-4f268\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.146778 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.146700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.146778 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.146761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.247442 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.247348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.247442 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.247410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f268\" (UniqueName: \"kubernetes.io/projected/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-api-access-4f268\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.247442 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.247439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.247763 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.247482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.247763 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.247532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/88647fa7-2e0a-4fa8-b77b-3f449a52d644-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.247914 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.247894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.248209 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.248189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.248257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.248206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.250007 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.249985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/88647fa7-2e0a-4fa8-b77b-3f449a52d644-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.256475 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.256454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f268\" (UniqueName: \"kubernetes.io/projected/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-api-access-4f268\") pod \"isvc-sklearn-batcher-predictor-78b5fd7767-47skv\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.304625 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.304593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:06.434696 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:06.434667    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"]
Jul 13 19:01:06.439214 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:01:06.439186    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88647fa7_2e0a_4fa8_b77b_3f449a52d644.slice/crio-a0d702d0e935c610b9b4936260e988f604118f89ab831ca286525501ed200a52 WatchSource:0}: Error finding container a0d702d0e935c610b9b4936260e988f604118f89ab831ca286525501ed200a52: Status 404 returned error can't find the container with id a0d702d0e935c610b9b4936260e988f604118f89ab831ca286525501ed200a52
Jul 13 19:01:07.132849 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:07.132811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerStarted","Data":"8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401"}
Jul 13 19:01:07.133235 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:07.132887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerStarted","Data":"a0d702d0e935c610b9b4936260e988f604118f89ab831ca286525501ed200a52"}
Jul 13 19:01:11.151622 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:11.151590    2581 generic.go:356] "Generic (PLEG): container finished" podID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerID="8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401" exitCode=0
Jul 13 19:01:11.152054 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:11.151631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerDied","Data":"8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401"}
Jul 13 19:01:26.230920 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:26.230886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerStarted","Data":"4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c"}
Jul 13 19:01:29.249373 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:29.249330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerStarted","Data":"aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13"}
Jul 13 19:01:31.261611 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:31.261573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerStarted","Data":"58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10"}
Jul 13 19:01:31.261983 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:31.261774    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:31.284535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:31.284483    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podStartSLOduration=2.747433786 podStartE2EDuration="26.284468714s" totalImagesPullingTime="19.887482594s" totalInitContainerRuntime="3.649552334s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:01:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:01:31.282140863 +0000 UTC m=+660.329336018" watchObservedRunningTime="2026-07-13 19:01:31.284468714 +0000 UTC m=+660.331663869"
Jul 13 19:01:32.266269 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:32.266238    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:32.266760 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:32.266304    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:32.267835 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:32.267794    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:01:32.268423 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:32.268398    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:01:33.270781 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:33.270738    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:01:33.271317 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:33.271291    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:01:33.274961 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:33.274941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:01:34.274759 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:34.274719    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:01:34.275205 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:34.274969    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:01:44.275337 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:44.275287    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:01:44.275790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:44.275693    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:01:54.275356 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:54.275309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:01:54.275846 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:01:54.275824    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:04.274898 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:04.274844    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:02:04.275353 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:04.275248    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:14.275133 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:14.275087    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:02:14.275611 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:14.275585    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:24.275362 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:24.275267    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:02:24.275862 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:24.275655    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:34.275074 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:34.275017    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:02:34.275481 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:34.275405    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:44.275454 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:44.275420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:02:44.275897 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:44.275746    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:02:50.913712 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:50.913677    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"]
Jul 13 19:02:50.914139 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:50.913982    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" containerID="cri-o://4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c" gracePeriod=30
Jul 13 19:02:50.914139 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:50.914019    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" containerID="cri-o://58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10" gracePeriod=30
Jul 13 19:02:50.914253 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:50.914161    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" containerID="cri-o://aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13" gracePeriod=30
Jul 13 19:02:51.025499 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.025463    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"]
Jul 13 19:02:51.028317 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.028295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.030155 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.030131    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 13 19:02:51.030256 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.030185    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 13 19:02:51.042642 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.042622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"]
Jul 13 19:02:51.076084 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.076053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/08d3027c-9641-4933-aa6b-11d4ccf437c3-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.076084 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.076087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.076293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.076109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gf4p\" (UniqueName: \"kubernetes.io/projected/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-api-access-8gf4p\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.076293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.076201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.076293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.076247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08d3027c-9641-4933-aa6b-11d4ccf437c3-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.176970 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.176885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.176970 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.176934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08d3027c-9641-4933-aa6b-11d4ccf437c3-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.177243 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.177005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/08d3027c-9641-4933-aa6b-11d4ccf437c3-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.177243 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.177033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.177243 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.177058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gf4p\" (UniqueName: \"kubernetes.io/projected/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-api-access-8gf4p\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.177576 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.177532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/08d3027c-9641-4933-aa6b-11d4ccf437c3-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.177750 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.177726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.177810 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.177726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.179331 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.179313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08d3027c-9641-4933-aa6b-11d4ccf437c3-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.184209 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.184192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gf4p\" (UniqueName: \"kubernetes.io/projected/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-api-access-8gf4p\") pod \"isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.338293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.338261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:51.466079 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.465973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"]
Jul 13 19:02:51.468227 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:02:51.468201    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d3027c_9641_4933_aa6b_11d4ccf437c3.slice/crio-bbf9a52a33eeaaab7d6271128dbad7e6318d139817fef053fa2d7a9032ac82bb WatchSource:0}: Error finding container bbf9a52a33eeaaab7d6271128dbad7e6318d139817fef053fa2d7a9032ac82bb: Status 404 returned error can't find the container with id bbf9a52a33eeaaab7d6271128dbad7e6318d139817fef053fa2d7a9032ac82bb
Jul 13 19:02:51.606364 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.606332    2581 generic.go:356] "Generic (PLEG): container finished" podID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerID="aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13" exitCode=2
Jul 13 19:02:51.606531 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.606401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerDied","Data":"aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13"}
Jul 13 19:02:51.608290 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.608267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerStarted","Data":"16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9"}
Jul 13 19:02:51.608410 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:51.608326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerStarted","Data":"bbf9a52a33eeaaab7d6271128dbad7e6318d139817fef053fa2d7a9032ac82bb"}
Jul 13 19:02:53.271100 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:53.271057    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 13 19:02:54.275236 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:54.275186    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:02:54.275641 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:54.275517    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:55.628744 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:55.628713    2581 generic.go:356] "Generic (PLEG): container finished" podID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerID="4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c" exitCode=0
Jul 13 19:02:55.629166 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:55.628779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerDied","Data":"4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c"}
Jul 13 19:02:55.630312 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:55.630287    2581 generic.go:356] "Generic (PLEG): container finished" podID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerID="16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9" exitCode=0
Jul 13 19:02:55.630419 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:55.630347    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerDied","Data":"16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9"}
Jul 13 19:02:56.636838 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.636803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerStarted","Data":"06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec"}
Jul 13 19:02:56.637302 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.636885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerStarted","Data":"fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9"}
Jul 13 19:02:56.637302 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.636927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerStarted","Data":"459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e"}
Jul 13 19:02:56.637302 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.637249    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:56.637458 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.637315    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:56.637458 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.637350    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:02:56.638792 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.638760    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:02:56.639408 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.639379    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:56.658896 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:56.658842    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podStartSLOduration=2.01617828 podStartE2EDuration="5.658829408s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.642651128s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:02:56.65663933 +0000 UTC m=+745.703834484" watchObservedRunningTime="2026-07-13 19:02:56.658829408 +0000 UTC m=+745.706024621"
Jul 13 19:02:57.641622 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:57.641580    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:02:57.642069 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:57.642044    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:02:58.270976 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:02:58.270934    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 13 19:03:02.646493 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:02.646463    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:03:02.647041 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:02.647015    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:03:02.647336 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:02.647309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:03.271370 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:03.271331    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 13 19:03:03.271540 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:03.271460    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:03:04.274904 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:04.274850    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:03:04.275299 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:04.275174    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:08.271096 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:08.271052    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 13 19:03:12.646957 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:12.646914    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:03:12.647397 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:12.647374    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:13.271100 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:13.271055    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 13 19:03:14.274691 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:14.274649    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Jul 13 19:03:14.275089 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:14.274769    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:03:14.275089 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:14.274915    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:14.275089 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:14.275033    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:03:18.271902 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:18.271864    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 13 19:03:21.064615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.064586    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:03:21.142039 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-rbac-proxy-sar-config\") pod \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") "
Jul 13 19:03:21.142208 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142079    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kserve-provision-location\") pod \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") "
Jul 13 19:03:21.142208 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142189    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/88647fa7-2e0a-4fa8-b77b-3f449a52d644-proxy-tls\") pod \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") "
Jul 13 19:03:21.142313 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-cabundle-cert\") pod \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") "
Jul 13 19:03:21.142313 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f268\" (UniqueName: \"kubernetes.io/projected/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-api-access-4f268\") pod \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\" (UID: \"88647fa7-2e0a-4fa8-b77b-3f449a52d644\") "
Jul 13 19:03:21.142424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142350    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "88647fa7-2e0a-4fa8-b77b-3f449a52d644" (UID: "88647fa7-2e0a-4fa8-b77b-3f449a52d644"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:03:21.142424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "88647fa7-2e0a-4fa8-b77b-3f449a52d644" (UID: "88647fa7-2e0a-4fa8-b77b-3f449a52d644"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:03:21.142613 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142593    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:03:21.142709 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142613    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "88647fa7-2e0a-4fa8-b77b-3f449a52d644" (UID: "88647fa7-2e0a-4fa8-b77b-3f449a52d644"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:03:21.142709 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.142618    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:03:21.144441 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.144409    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88647fa7-2e0a-4fa8-b77b-3f449a52d644-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "88647fa7-2e0a-4fa8-b77b-3f449a52d644" (UID: "88647fa7-2e0a-4fa8-b77b-3f449a52d644"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:03:21.144535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.144448    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-api-access-4f268" (OuterVolumeSpecName: "kube-api-access-4f268") pod "88647fa7-2e0a-4fa8-b77b-3f449a52d644" (UID: "88647fa7-2e0a-4fa8-b77b-3f449a52d644"). InnerVolumeSpecName "kube-api-access-4f268". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:03:21.243847 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.243767    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f268\" (UniqueName: \"kubernetes.io/projected/88647fa7-2e0a-4fa8-b77b-3f449a52d644-kube-api-access-4f268\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:03:21.243847 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.243795    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/88647fa7-2e0a-4fa8-b77b-3f449a52d644-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:03:21.243847 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.243806    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/88647fa7-2e0a-4fa8-b77b-3f449a52d644-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:03:21.749026 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.748989    2581 generic.go:356] "Generic (PLEG): container finished" podID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerID="58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10" exitCode=0
Jul 13 19:03:21.749171 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.749072    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"
Jul 13 19:03:21.749171 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.749072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerDied","Data":"58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10"}
Jul 13 19:03:21.749171 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.749164    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv" event={"ID":"88647fa7-2e0a-4fa8-b77b-3f449a52d644","Type":"ContainerDied","Data":"a0d702d0e935c610b9b4936260e988f604118f89ab831ca286525501ed200a52"}
Jul 13 19:03:21.749297 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.749196    2581 scope.go:117] "RemoveContainer" containerID="58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10"
Jul 13 19:03:21.757443 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.757426    2581 scope.go:117] "RemoveContainer" containerID="aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13"
Jul 13 19:03:21.764807 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.764791    2581 scope.go:117] "RemoveContainer" containerID="4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c"
Jul 13 19:03:21.768086 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.768065    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"]
Jul 13 19:03:21.772990 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.772973    2581 scope.go:117] "RemoveContainer" containerID="8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401"
Jul 13 19:03:21.773810 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.773714    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-78b5fd7767-47skv"]
Jul 13 19:03:21.780133 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780105    2581 scope.go:117] "RemoveContainer" containerID="58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10"
Jul 13 19:03:21.780399 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:03:21.780380    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10\": container with ID starting with 58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10 not found: ID does not exist" containerID="58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10"
Jul 13 19:03:21.780451 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780407    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10"} err="failed to get container status \"58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10\": rpc error: code = NotFound desc = could not find container \"58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10\": container with ID starting with 58a1da83761cf00223ebed133dc6413bf12f0db88d84870843f569e69ce83f10 not found: ID does not exist"
Jul 13 19:03:21.780451 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780425    2581 scope.go:117] "RemoveContainer" containerID="aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13"
Jul 13 19:03:21.780681 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:03:21.780666    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13\": container with ID starting with aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13 not found: ID does not exist" containerID="aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13"
Jul 13 19:03:21.780732 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780683    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13"} err="failed to get container status \"aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13\": rpc error: code = NotFound desc = could not find container \"aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13\": container with ID starting with aa9d5800d24a03813377df21f5455789d0285002642dfd57f8130c6b373d5c13 not found: ID does not exist"
Jul 13 19:03:21.780732 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780695    2581 scope.go:117] "RemoveContainer" containerID="4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c"
Jul 13 19:03:21.780905 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:03:21.780889    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c\": container with ID starting with 4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c not found: ID does not exist" containerID="4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c"
Jul 13 19:03:21.780965 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780911    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c"} err="failed to get container status \"4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c\": rpc error: code = NotFound desc = could not find container \"4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c\": container with ID starting with 4bf5d0de3903be2f09781870523ee0801ad78efead4c903e4d129b1faba7d04c not found: ID does not exist"
Jul 13 19:03:21.780965 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.780926    2581 scope.go:117] "RemoveContainer" containerID="8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401"
Jul 13 19:03:21.781150 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:03:21.781122    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401\": container with ID starting with 8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401 not found: ID does not exist" containerID="8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401"
Jul 13 19:03:21.781241 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:21.781154    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401"} err="failed to get container status \"8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401\": rpc error: code = NotFound desc = could not find container \"8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401\": container with ID starting with 8205448701ce4edb8eafdcc30fa0e93dda611fa7364094156ac0a7c5be39a401 not found: ID does not exist"
Jul 13 19:03:22.647219 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:22.647176    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:03:22.647691 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:22.647672    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:23.506811 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:23.506777    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" path="/var/lib/kubelet/pods/88647fa7-2e0a-4fa8-b77b-3f449a52d644/volumes"
Jul 13 19:03:32.647697 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:32.647657    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:03:32.648216 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:32.648192    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:42.647016 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:42.646974    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:03:42.647463 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:42.647391    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:03:52.646967 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:52.646925    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:03:52.647395 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:03:52.647297    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:02.646984 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:02.646938    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:04:02.647466 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:02.647284    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:09.507245 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:09.507219    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:09.507702 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:09.507288    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:16.145919 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.145886    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"]
Jul 13 19:04:16.146408 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.146135    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" containerID="cri-o://459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e" gracePeriod=30
Jul 13 19:04:16.146408 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.146170    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" containerID="cri-o://06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec" gracePeriod=30
Jul 13 19:04:16.146408 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.146194    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" containerID="cri-o://fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9" gracePeriod=30
Jul 13 19:04:16.229701 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.229663    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"]
Jul 13 19:04:16.230076 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230055    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy"
Jul 13 19:04:16.230076 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230074    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy"
Jul 13 19:04:16.230225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230085    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent"
Jul 13 19:04:16.230225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230093    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent"
Jul 13 19:04:16.230225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230115    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container"
Jul 13 19:04:16.230225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230124    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container"
Jul 13 19:04:16.230225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230140    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="storage-initializer"
Jul 13 19:04:16.230225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230148    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="storage-initializer"
Jul 13 19:04:16.230592 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230234    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="agent"
Jul 13 19:04:16.230592 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230249    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kserve-container"
Jul 13 19:04:16.230592 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.230260    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88647fa7-2e0a-4fa8-b77b-3f449a52d644" containerName="kube-rbac-proxy"
Jul 13 19:04:16.233327 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.233306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.237853 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.237825    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 13 19:04:16.237853 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.237843    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 13 19:04:16.246045 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.246023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"]
Jul 13 19:04:16.421785 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.421693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fcc110fb-2fd1-40d5-b6c7-16462b777aba-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.421785 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.421746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg2jq\" (UniqueName: \"kubernetes.io/projected/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-api-access-rg2jq\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.421971 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.421841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.523180 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.523133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fcc110fb-2fd1-40d5-b6c7-16462b777aba-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.523374 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.523227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg2jq\" (UniqueName: \"kubernetes.io/projected/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-api-access-rg2jq\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.523374 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.523297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.524061 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.524030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.525588 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.525550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fcc110fb-2fd1-40d5-b6c7-16462b777aba-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.547702 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.547676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg2jq\" (UniqueName: \"kubernetes.io/projected/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-api-access-rg2jq\") pod \"message-dumper-predictor-6575bcfdfd-v85ks\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.843966 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.843876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:16.977426 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.977401    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"]
Jul 13 19:04:16.979737 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:04:16.979712    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc110fb_2fd1_40d5_b6c7_16462b777aba.slice/crio-b876652bccd23a7e78e2a1f2847f37d3cd1749d7ac6f04fd3a9f1a4a75ac7e96 WatchSource:0}: Error finding container b876652bccd23a7e78e2a1f2847f37d3cd1749d7ac6f04fd3a9f1a4a75ac7e96: Status 404 returned error can't find the container with id b876652bccd23a7e78e2a1f2847f37d3cd1749d7ac6f04fd3a9f1a4a75ac7e96
Jul 13 19:04:16.981612 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.981595    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:04:16.988603 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.988581    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" event={"ID":"fcc110fb-2fd1-40d5-b6c7-16462b777aba","Type":"ContainerStarted","Data":"b876652bccd23a7e78e2a1f2847f37d3cd1749d7ac6f04fd3a9f1a4a75ac7e96"}
Jul 13 19:04:16.992300 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.992279    2581 generic.go:356] "Generic (PLEG): container finished" podID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerID="fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9" exitCode=2
Jul 13 19:04:16.992381 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:16.992339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerDied","Data":"fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9"}
Jul 13 19:04:17.642611 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:17.642532    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 13 19:04:19.003686 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.003651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" event={"ID":"fcc110fb-2fd1-40d5-b6c7-16462b777aba","Type":"ContainerStarted","Data":"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"}
Jul 13 19:04:19.004079 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.003707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" event={"ID":"fcc110fb-2fd1-40d5-b6c7-16462b777aba","Type":"ContainerStarted","Data":"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"}
Jul 13 19:04:19.004079 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.003731    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:19.004079 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.003796    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:19.005579 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.005547    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:19.024828 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.024781    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" podStartSLOduration=1.880405565 podStartE2EDuration="3.024769019s" totalImagesPullingTime="1.144363454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:04:19.023274052 +0000 UTC m=+828.070469209" watchObservedRunningTime="2026-07-13 19:04:19.024769019 +0000 UTC m=+828.071964174"
Jul 13 19:04:19.502265 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.502230    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:04:19.502693 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:19.502669    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:21.015726 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:21.015641    2581 generic.go:356] "Generic (PLEG): container finished" podID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerID="459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e" exitCode=0
Jul 13 19:04:21.015726 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:21.015712    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerDied","Data":"459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e"}
Jul 13 19:04:22.642313 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:22.642270    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 13 19:04:26.020338 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:26.020311    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:04:27.641978 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:27.641934    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 13 19:04:27.642442 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:27.642068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:29.502838 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:29.502802    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:04:29.503260 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:29.503200    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:32.642796 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:32.642748    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 13 19:04:36.311632 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.311594    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"]
Jul 13 19:04:36.315355 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.315336    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.317843 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.317820    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 13 19:04:36.318467 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.318452    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 13 19:04:36.327803 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.327781    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"]
Jul 13 19:04:36.388164 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.388136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56458e18-dcac-4299-8610-9015087e177e-kserve-provision-location\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.388323 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.388173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.388323 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.388270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56458e18-dcac-4299-8610-9015087e177e-proxy-tls\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.388323 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.388297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4xrb\" (UniqueName: \"kubernetes.io/projected/56458e18-dcac-4299-8610-9015087e177e-kube-api-access-z4xrb\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.388434 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.388330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-cabundle-cert\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.489475 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.489428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56458e18-dcac-4299-8610-9015087e177e-proxy-tls\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.489475 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.489475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4xrb\" (UniqueName: \"kubernetes.io/projected/56458e18-dcac-4299-8610-9015087e177e-kube-api-access-z4xrb\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.489700 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.489505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-cabundle-cert\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.489700 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.489588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56458e18-dcac-4299-8610-9015087e177e-kserve-provision-location\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.489700 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.489629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.490076 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.490046    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56458e18-dcac-4299-8610-9015087e177e-kserve-provision-location\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.490312 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.490293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.490354 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.490315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-cabundle-cert\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.491894 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.491873    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56458e18-dcac-4299-8610-9015087e177e-proxy-tls\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.497630 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.497607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4xrb\" (UniqueName: \"kubernetes.io/projected/56458e18-dcac-4299-8610-9015087e177e-kube-api-access-z4xrb\") pod \"isvc-logger-predictor-5d9d8fb8b4-c549m\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.626623 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.626525    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:36.762291 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:36.762269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"]
Jul 13 19:04:36.764754 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:04:36.764728    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56458e18_dcac_4299_8610_9015087e177e.slice/crio-f769471eb6074a012606358cbee926436ddb5f8ea73615dbc2bd2231b8ec4922 WatchSource:0}: Error finding container f769471eb6074a012606358cbee926436ddb5f8ea73615dbc2bd2231b8ec4922: Status 404 returned error can't find the container with id f769471eb6074a012606358cbee926436ddb5f8ea73615dbc2bd2231b8ec4922
Jul 13 19:04:37.086531 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:37.086492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerStarted","Data":"8043faf666481102120e128b12c8ba9ba90a652cc9ebfba56846e548c566abf4"}
Jul 13 19:04:37.086734 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:37.086592    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerStarted","Data":"f769471eb6074a012606358cbee926436ddb5f8ea73615dbc2bd2231b8ec4922"}
Jul 13 19:04:37.642506 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:37.642467    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 13 19:04:39.502480 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:39.502431    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:5000: connect: connection refused"
Jul 13 19:04:39.502936 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:39.502732    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:39.507149 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:39.507129    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:39.507287 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:39.507196    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:41.105927 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:41.105845    2581 generic.go:356] "Generic (PLEG): container finished" podID="56458e18-dcac-4299-8610-9015087e177e" containerID="8043faf666481102120e128b12c8ba9ba90a652cc9ebfba56846e548c566abf4" exitCode=0
Jul 13 19:04:41.106260 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:41.105922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerDied","Data":"8043faf666481102120e128b12c8ba9ba90a652cc9ebfba56846e548c566abf4"}
Jul 13 19:04:42.112927 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:42.112893    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerStarted","Data":"8b36bcf60fabd2efd3ef7132c96133bb8a5abc199b30b23c304acac1974ef4de"}
Jul 13 19:04:42.113296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:42.112960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerStarted","Data":"1495f758489616312ce20483148a56d8fcb47366f840edc0c8282eb5266ab8c8"}
Jul 13 19:04:42.113296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:42.112988    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerStarted","Data":"7d2ae749714de6bf10b855989be4d54acb6bdd84d6ff1b0192d162636f97c251"}
Jul 13 19:04:42.113296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:42.113190    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:42.145385 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:42.145323    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podStartSLOduration=2.268864716 podStartE2EDuration="6.14530304s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.876438324s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:04:42.143864528 +0000 UTC m=+851.191059683" watchObservedRunningTime="2026-07-13 19:04:42.14530304 +0000 UTC m=+851.192498200"
Jul 13 19:04:42.642462 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:42.642419    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 13 19:04:43.117369 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:43.117285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:43.117369 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:43.117346    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:43.118640 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:43.118615    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:04:43.119255 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:43.119231    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:44.122117 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:44.122065    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:04:44.122573 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:44.122528    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:46.302507 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.302480    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:46.373928 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.373898    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-rbac-proxy-sar-config\") pod \"08d3027c-9641-4933-aa6b-11d4ccf437c3\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") "
Jul 13 19:04:46.374110 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.373955    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08d3027c-9641-4933-aa6b-11d4ccf437c3-proxy-tls\") pod \"08d3027c-9641-4933-aa6b-11d4ccf437c3\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") "
Jul 13 19:04:46.374110 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.373979    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gf4p\" (UniqueName: \"kubernetes.io/projected/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-api-access-8gf4p\") pod \"08d3027c-9641-4933-aa6b-11d4ccf437c3\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") "
Jul 13 19:04:46.374110 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.374009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/08d3027c-9641-4933-aa6b-11d4ccf437c3-kserve-provision-location\") pod \"08d3027c-9641-4933-aa6b-11d4ccf437c3\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") "
Jul 13 19:04:46.374110 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.374044    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-cabundle-cert\") pod \"08d3027c-9641-4933-aa6b-11d4ccf437c3\" (UID: \"08d3027c-9641-4933-aa6b-11d4ccf437c3\") "
Jul 13 19:04:46.374323 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.374295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "08d3027c-9641-4933-aa6b-11d4ccf437c3" (UID: "08d3027c-9641-4933-aa6b-11d4ccf437c3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:04:46.374376 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.374322    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d3027c-9641-4933-aa6b-11d4ccf437c3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "08d3027c-9641-4933-aa6b-11d4ccf437c3" (UID: "08d3027c-9641-4933-aa6b-11d4ccf437c3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:04:46.374475 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.374453    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "08d3027c-9641-4933-aa6b-11d4ccf437c3" (UID: "08d3027c-9641-4933-aa6b-11d4ccf437c3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:04:46.376059 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.376033    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-api-access-8gf4p" (OuterVolumeSpecName: "kube-api-access-8gf4p") pod "08d3027c-9641-4933-aa6b-11d4ccf437c3" (UID: "08d3027c-9641-4933-aa6b-11d4ccf437c3"). InnerVolumeSpecName "kube-api-access-8gf4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:04:46.376142 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.376041    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08d3027c-9641-4933-aa6b-11d4ccf437c3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "08d3027c-9641-4933-aa6b-11d4ccf437c3" (UID: "08d3027c-9641-4933-aa6b-11d4ccf437c3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:04:46.475257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.475183    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:04:46.475257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.475209    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08d3027c-9641-4933-aa6b-11d4ccf437c3-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:04:46.475257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.475237    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gf4p\" (UniqueName: \"kubernetes.io/projected/08d3027c-9641-4933-aa6b-11d4ccf437c3-kube-api-access-8gf4p\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:04:46.475257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.475246    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/08d3027c-9641-4933-aa6b-11d4ccf437c3-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:04:46.475257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:46.475255    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/08d3027c-9641-4933-aa6b-11d4ccf437c3-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:04:47.138393 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.138352    2581 generic.go:356] "Generic (PLEG): container finished" podID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerID="06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec" exitCode=0
Jul 13 19:04:47.138548 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.138420    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"
Jul 13 19:04:47.138548 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.138428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerDied","Data":"06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec"}
Jul 13 19:04:47.138548 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.138491    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2" event={"ID":"08d3027c-9641-4933-aa6b-11d4ccf437c3","Type":"ContainerDied","Data":"bbf9a52a33eeaaab7d6271128dbad7e6318d139817fef053fa2d7a9032ac82bb"}
Jul 13 19:04:47.138548 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.138524    2581 scope.go:117] "RemoveContainer" containerID="06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec"
Jul 13 19:04:47.149292 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.149271    2581 scope.go:117] "RemoveContainer" containerID="fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9"
Jul 13 19:04:47.157321 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.157305    2581 scope.go:117] "RemoveContainer" containerID="459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e"
Jul 13 19:04:47.164386 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.164371    2581 scope.go:117] "RemoveContainer" containerID="16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9"
Jul 13 19:04:47.168909 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.168871    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"]
Jul 13 19:04:47.171529 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.171487    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5f9bdc45b6-5zmw2"]
Jul 13 19:04:47.173351 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.173337    2581 scope.go:117] "RemoveContainer" containerID="06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec"
Jul 13 19:04:47.173630 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:04:47.173611    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec\": container with ID starting with 06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec not found: ID does not exist" containerID="06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec"
Jul 13 19:04:47.173710 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.173652    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec"} err="failed to get container status \"06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec\": rpc error: code = NotFound desc = could not find container \"06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec\": container with ID starting with 06d106c2c73c7bddbd759da6bf453323b15ac0b677a699c76d41f5656dd7c3ec not found: ID does not exist"
Jul 13 19:04:47.173710 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.173677    2581 scope.go:117] "RemoveContainer" containerID="fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9"
Jul 13 19:04:47.173971 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:04:47.173952    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9\": container with ID starting with fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9 not found: ID does not exist" containerID="fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9"
Jul 13 19:04:47.174017 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.173977    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9"} err="failed to get container status \"fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9\": rpc error: code = NotFound desc = could not find container \"fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9\": container with ID starting with fd0b9dfa8adaf1aa3dace890f8d9d70704bbfecae6f5dda13303290b953a65f9 not found: ID does not exist"
Jul 13 19:04:47.174017 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.173992    2581 scope.go:117] "RemoveContainer" containerID="459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e"
Jul 13 19:04:47.174214 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:04:47.174198    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e\": container with ID starting with 459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e not found: ID does not exist" containerID="459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e"
Jul 13 19:04:47.174257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.174219    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e"} err="failed to get container status \"459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e\": rpc error: code = NotFound desc = could not find container \"459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e\": container with ID starting with 459f475f460ef6fecaf721a279c875e260ad146de6fdb23432478d730f23274e not found: ID does not exist"
Jul 13 19:04:47.174257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.174233    2581 scope.go:117] "RemoveContainer" containerID="16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9"
Jul 13 19:04:47.174430 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:04:47.174416    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9\": container with ID starting with 16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9 not found: ID does not exist" containerID="16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9"
Jul 13 19:04:47.174467 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.174432    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9"} err="failed to get container status \"16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9\": rpc error: code = NotFound desc = could not find container \"16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9\": container with ID starting with 16c1c38c88ef7b5c43d751749f7702f57cd9172a3c5e15383dcf31a718fa89e9 not found: ID does not exist"
Jul 13 19:04:47.506452 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:47.506376    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" path="/var/lib/kubelet/pods/08d3027c-9641-4933-aa6b-11d4ccf437c3/volumes"
Jul 13 19:04:49.126609 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:49.126573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:04:49.127263 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:49.127232    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:04:49.127492 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:49.127459    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:04:59.128004 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:59.127955    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:04:59.128535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:04:59.128420    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:05:09.127692 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:09.127645    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:05:09.128133 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:09.128110    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:05:19.127171 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:19.127119    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:05:19.127605 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:19.127488    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:05:29.127860 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:29.127760    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:05:29.128455 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:29.128425    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:05:31.401106 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:31.401079    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:05:31.401545 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:31.401487    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:05:39.127211 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:39.127169    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:05:39.127693 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:39.127670    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:05:49.127619 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:49.127580    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:05:49.128115 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:49.128062    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:05:59.128291 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:59.128262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:05:59.128711 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:05:59.128636    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:06:11.306547 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.306516    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-v85ks_fcc110fb-2fd1-40d5-b6c7-16462b777aba/kserve-container/0.log"
Jul 13 19:06:11.543365 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.543329    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"]
Jul 13 19:06:11.543625 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.543590    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" containerID="cri-o://7d2ae749714de6bf10b855989be4d54acb6bdd84d6ff1b0192d162636f97c251" gracePeriod=30
Jul 13 19:06:11.543625 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.543614    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" containerID="cri-o://8b36bcf60fabd2efd3ef7132c96133bb8a5abc199b30b23c304acac1974ef4de" gracePeriod=30
Jul 13 19:06:11.543829 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.543647    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" containerID="cri-o://1495f758489616312ce20483148a56d8fcb47366f840edc0c8282eb5266ab8c8" gracePeriod=30
Jul 13 19:06:11.653256 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.653226    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"]
Jul 13 19:06:11.653509 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.653460    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kserve-container" containerID="cri-o://bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb" gracePeriod=30
Jul 13 19:06:11.653735 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.653508    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kube-rbac-proxy" containerID="cri-o://6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b" gracePeriod=30
Jul 13 19:06:11.716934 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.716907    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"]
Jul 13 19:06:11.717254 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717243    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container"
Jul 13 19:06:11.717296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717255    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container"
Jul 13 19:06:11.717296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717268    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy"
Jul 13 19:06:11.717296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717274    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy"
Jul 13 19:06:11.717296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717288    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="storage-initializer"
Jul 13 19:06:11.717296 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717294    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="storage-initializer"
Jul 13 19:06:11.717441 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717306    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent"
Jul 13 19:06:11.717441 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717312    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent"
Jul 13 19:06:11.717441 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717367    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kserve-container"
Jul 13 19:06:11.717441 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717377    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="agent"
Jul 13 19:06:11.717441 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.717386    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d3027c-9641-4933-aa6b-11d4ccf437c3" containerName="kube-rbac-proxy"
Jul 13 19:06:11.720373 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.720357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.722243 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.722222    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 13 19:06:11.722346 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.722232    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 13 19:06:11.734914 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.734892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"]
Jul 13 19:06:11.822117 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.822081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn2h9\" (UniqueName: \"kubernetes.io/projected/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-api-access-sn2h9\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.822253 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.822138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d09d0c06-ecac-4b00-b828-3bd74278d11b-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.822253 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.822217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.822380 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.822265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.822380 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.822311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.898604 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.898580    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:06:11.923594 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.923504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn2h9\" (UniqueName: \"kubernetes.io/projected/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-api-access-sn2h9\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.923594 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.923546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d09d0c06-ecac-4b00-b828-3bd74278d11b-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.923594 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.923585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.924424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.923647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.924424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.923694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.924424 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:06:11.923872    2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v1-predictor-serving-cert: secret "isvc-autogluon-v1-predictor-serving-cert" not found
Jul 13 19:06:11.924424 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:06:11.923957    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls podName:d09d0c06-ecac-4b00-b828-3bd74278d11b nodeName:}" failed. No retries permitted until 2026-07-13 19:06:12.423936903 +0000 UTC m=+941.471132048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls") pod "isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" (UID: "d09d0c06-ecac-4b00-b828-3bd74278d11b") : secret "isvc-autogluon-v1-predictor-serving-cert" not found
Jul 13 19:06:11.924424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.923992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d09d0c06-ecac-4b00-b828-3bd74278d11b-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.924424 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.924369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.924815 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.924436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:11.932277 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:11.932251    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn2h9\" (UniqueName: \"kubernetes.io/projected/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-api-access-sn2h9\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:12.024977 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.024949    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-rbac-proxy-sar-config\") pod \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") "
Jul 13 19:06:12.025137 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.025047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fcc110fb-2fd1-40d5-b6c7-16462b777aba-proxy-tls\") pod \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") "
Jul 13 19:06:12.025137 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.025070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg2jq\" (UniqueName: \"kubernetes.io/projected/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-api-access-rg2jq\") pod \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\" (UID: \"fcc110fb-2fd1-40d5-b6c7-16462b777aba\") "
Jul 13 19:06:12.025302 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.025279    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fcc110fb-2fd1-40d5-b6c7-16462b777aba" (UID: "fcc110fb-2fd1-40d5-b6c7-16462b777aba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:06:12.025381 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.025364    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:12.027227 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.027201    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc110fb-2fd1-40d5-b6c7-16462b777aba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fcc110fb-2fd1-40d5-b6c7-16462b777aba" (UID: "fcc110fb-2fd1-40d5-b6c7-16462b777aba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:06:12.027351 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.027270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-api-access-rg2jq" (OuterVolumeSpecName: "kube-api-access-rg2jq") pod "fcc110fb-2fd1-40d5-b6c7-16462b777aba" (UID: "fcc110fb-2fd1-40d5-b6c7-16462b777aba"). InnerVolumeSpecName "kube-api-access-rg2jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:06:12.126358 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.126320    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fcc110fb-2fd1-40d5-b6c7-16462b777aba-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:12.126358 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.126351    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg2jq\" (UniqueName: \"kubernetes.io/projected/fcc110fb-2fd1-40d5-b6c7-16462b777aba-kube-api-access-rg2jq\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:12.428598 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.428535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:12.431167 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.431139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:12.531528 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.531491    2581 generic.go:356] "Generic (PLEG): container finished" podID="56458e18-dcac-4299-8610-9015087e177e" containerID="1495f758489616312ce20483148a56d8fcb47366f840edc0c8282eb5266ab8c8" exitCode=2
Jul 13 19:06:12.531704 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.531581    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerDied","Data":"1495f758489616312ce20483148a56d8fcb47366f840edc0c8282eb5266ab8c8"}
Jul 13 19:06:12.533041 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533019    2581 generic.go:356] "Generic (PLEG): container finished" podID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerID="6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b" exitCode=2
Jul 13 19:06:12.533041 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533038    2581 generic.go:356] "Generic (PLEG): container finished" podID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerID="bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb" exitCode=2
Jul 13 19:06:12.533213 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533081    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"
Jul 13 19:06:12.533213 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" event={"ID":"fcc110fb-2fd1-40d5-b6c7-16462b777aba","Type":"ContainerDied","Data":"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"}
Jul 13 19:06:12.533213 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" event={"ID":"fcc110fb-2fd1-40d5-b6c7-16462b777aba","Type":"ContainerDied","Data":"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"}
Jul 13 19:06:12.533213 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks" event={"ID":"fcc110fb-2fd1-40d5-b6c7-16462b777aba","Type":"ContainerDied","Data":"b876652bccd23a7e78e2a1f2847f37d3cd1749d7ac6f04fd3a9f1a4a75ac7e96"}
Jul 13 19:06:12.533213 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.533185    2581 scope.go:117] "RemoveContainer" containerID="6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"
Jul 13 19:06:12.542041 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.542026    2581 scope.go:117] "RemoveContainer" containerID="bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"
Jul 13 19:06:12.549653 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.549397    2581 scope.go:117] "RemoveContainer" containerID="6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"
Jul 13 19:06:12.549723 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:06:12.549696    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b\": container with ID starting with 6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b not found: ID does not exist" containerID="6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"
Jul 13 19:06:12.549778 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.549730    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"} err="failed to get container status \"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b\": rpc error: code = NotFound desc = could not find container \"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b\": container with ID starting with 6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b not found: ID does not exist"
Jul 13 19:06:12.549778 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.549754    2581 scope.go:117] "RemoveContainer" containerID="bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"
Jul 13 19:06:12.550001 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:06:12.549985    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb\": container with ID starting with bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb not found: ID does not exist" containerID="bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"
Jul 13 19:06:12.550043 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.550007    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"} err="failed to get container status \"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb\": rpc error: code = NotFound desc = could not find container \"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb\": container with ID starting with bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb not found: ID does not exist"
Jul 13 19:06:12.550043 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.550023    2581 scope.go:117] "RemoveContainer" containerID="6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"
Jul 13 19:06:12.550247 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.550227    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b"} err="failed to get container status \"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b\": rpc error: code = NotFound desc = could not find container \"6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b\": container with ID starting with 6fceb28f55b174197f287d0ee9740c4d7b911d7f79aee55d007ee2ae0c84356b not found: ID does not exist"
Jul 13 19:06:12.550292 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.550248    2581 scope.go:117] "RemoveContainer" containerID="bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"
Jul 13 19:06:12.550457 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.550441    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb"} err="failed to get container status \"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb\": rpc error: code = NotFound desc = could not find container \"bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb\": container with ID starting with bf339e263b4579bc2545d2cdb402bbedce6d0823dff8b020ace92986f97a73eb not found: ID does not exist"
Jul 13 19:06:12.555575 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.555544    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"]
Jul 13 19:06:12.560471 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.560453    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-v85ks"]
Jul 13 19:06:12.641868 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.641843    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:06:12.775773 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:12.775752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"]
Jul 13 19:06:12.778053 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:06:12.778026    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09d0c06_ecac_4b00_b828_3bd74278d11b.slice/crio-faa8f61c223f71e2d9bddf70286c28d29969334b599c04eeac8fac0da0dc7f08 WatchSource:0}: Error finding container faa8f61c223f71e2d9bddf70286c28d29969334b599c04eeac8fac0da0dc7f08: Status 404 returned error can't find the container with id faa8f61c223f71e2d9bddf70286c28d29969334b599c04eeac8fac0da0dc7f08
Jul 13 19:06:13.506491 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:13.506458    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" path="/var/lib/kubelet/pods/fcc110fb-2fd1-40d5-b6c7-16462b777aba/volumes"
Jul 13 19:06:13.539401 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:13.539370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerStarted","Data":"3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592"}
Jul 13 19:06:13.539568 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:13.539432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerStarted","Data":"faa8f61c223f71e2d9bddf70286c28d29969334b599c04eeac8fac0da0dc7f08"}
Jul 13 19:06:14.122670 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:14.122617    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 13 19:06:16.553794 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:16.553763    2581 generic.go:356] "Generic (PLEG): container finished" podID="56458e18-dcac-4299-8610-9015087e177e" containerID="7d2ae749714de6bf10b855989be4d54acb6bdd84d6ff1b0192d162636f97c251" exitCode=0
Jul 13 19:06:16.554175 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:16.553836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerDied","Data":"7d2ae749714de6bf10b855989be4d54acb6bdd84d6ff1b0192d162636f97c251"}
Jul 13 19:06:18.563946 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:18.563916    2581 generic.go:356] "Generic (PLEG): container finished" podID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerID="3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592" exitCode=0
Jul 13 19:06:18.564319 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:18.563960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerDied","Data":"3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592"}
Jul 13 19:06:19.122927 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:19.122876    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 13 19:06:19.127926 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:19.127858    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:06:19.129391 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:19.129328    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:06:24.123177 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:24.123127    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 13 19:06:24.123658 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:24.123266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:06:29.123250 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:29.123194    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 13 19:06:29.128054 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:29.128018    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:06:29.129290 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:29.129262    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:06:34.122316 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:34.122268    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 13 19:06:39.122654 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:39.122616    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 13 19:06:39.128445 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:39.128410    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:06:39.129223 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:39.129049    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:06:39.129895 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:39.129858    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 13 19:06:39.130076 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:39.130060    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:06:41.679194 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:41.679160    2581 generic.go:356] "Generic (PLEG): container finished" podID="56458e18-dcac-4299-8610-9015087e177e" containerID="8b36bcf60fabd2efd3ef7132c96133bb8a5abc199b30b23c304acac1974ef4de" exitCode=0
Jul 13 19:06:41.679578 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:41.679200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerDied","Data":"8b36bcf60fabd2efd3ef7132c96133bb8a5abc199b30b23c304acac1974ef4de"}
Jul 13 19:06:42.200188 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.200164    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:06:42.298113 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298083    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4xrb\" (UniqueName: \"kubernetes.io/projected/56458e18-dcac-4299-8610-9015087e177e-kube-api-access-z4xrb\") pod \"56458e18-dcac-4299-8610-9015087e177e\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") "
Jul 13 19:06:42.298280 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298122    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56458e18-dcac-4299-8610-9015087e177e-proxy-tls\") pod \"56458e18-dcac-4299-8610-9015087e177e\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") "
Jul 13 19:06:42.298280 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-cabundle-cert\") pod \"56458e18-dcac-4299-8610-9015087e177e\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") "
Jul 13 19:06:42.298403 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298315    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56458e18-dcac-4299-8610-9015087e177e-kserve-provision-location\") pod \"56458e18-dcac-4299-8610-9015087e177e\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") "
Jul 13 19:06:42.298403 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298359    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-kube-rbac-proxy-sar-config\") pod \"56458e18-dcac-4299-8610-9015087e177e\" (UID: \"56458e18-dcac-4299-8610-9015087e177e\") "
Jul 13 19:06:42.298724 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298678    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "56458e18-dcac-4299-8610-9015087e177e" (UID: "56458e18-dcac-4299-8610-9015087e177e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:06:42.298956 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56458e18-dcac-4299-8610-9015087e177e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "56458e18-dcac-4299-8610-9015087e177e" (UID: "56458e18-dcac-4299-8610-9015087e177e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:06:42.298956 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.298902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "56458e18-dcac-4299-8610-9015087e177e" (UID: "56458e18-dcac-4299-8610-9015087e177e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:06:42.300570 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.300537    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56458e18-dcac-4299-8610-9015087e177e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "56458e18-dcac-4299-8610-9015087e177e" (UID: "56458e18-dcac-4299-8610-9015087e177e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:06:42.300679 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.300641    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56458e18-dcac-4299-8610-9015087e177e-kube-api-access-z4xrb" (OuterVolumeSpecName: "kube-api-access-z4xrb") pod "56458e18-dcac-4299-8610-9015087e177e" (UID: "56458e18-dcac-4299-8610-9015087e177e"). InnerVolumeSpecName "kube-api-access-z4xrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:06:42.399730 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.399697    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:42.399730 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.399727    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56458e18-dcac-4299-8610-9015087e177e-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:42.399730 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.399736    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56458e18-dcac-4299-8610-9015087e177e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:42.399730 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.399745    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4xrb\" (UniqueName: \"kubernetes.io/projected/56458e18-dcac-4299-8610-9015087e177e-kube-api-access-z4xrb\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:42.400031 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.399753    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56458e18-dcac-4299-8610-9015087e177e-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:06:42.686583 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.686488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m" event={"ID":"56458e18-dcac-4299-8610-9015087e177e","Type":"ContainerDied","Data":"f769471eb6074a012606358cbee926436ddb5f8ea73615dbc2bd2231b8ec4922"}
Jul 13 19:06:42.687022 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.686581    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"
Jul 13 19:06:42.687022 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.686589    2581 scope.go:117] "RemoveContainer" containerID="8b36bcf60fabd2efd3ef7132c96133bb8a5abc199b30b23c304acac1974ef4de"
Jul 13 19:06:42.696964 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.696481    2581 scope.go:117] "RemoveContainer" containerID="1495f758489616312ce20483148a56d8fcb47366f840edc0c8282eb5266ab8c8"
Jul 13 19:06:42.706365 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.706349    2581 scope.go:117] "RemoveContainer" containerID="7d2ae749714de6bf10b855989be4d54acb6bdd84d6ff1b0192d162636f97c251"
Jul 13 19:06:42.714816 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.714783    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"]
Jul 13 19:06:42.716568 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.716532    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5d9d8fb8b4-c549m"]
Jul 13 19:06:42.716930 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:42.716912    2581 scope.go:117] "RemoveContainer" containerID="8043faf666481102120e128b12c8ba9ba90a652cc9ebfba56846e548c566abf4"
Jul 13 19:06:43.506891 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:06:43.506852    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56458e18-dcac-4299-8610-9015087e177e" path="/var/lib/kubelet/pods/56458e18-dcac-4299-8610-9015087e177e/volumes"
Jul 13 19:08:39.244848 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:39.244808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerStarted","Data":"57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385"}
Jul 13 19:08:39.245307 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:39.244874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerStarted","Data":"6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d"}
Jul 13 19:08:39.245307 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:39.244968    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:08:39.280999 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:39.280938    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" podStartSLOduration=3.316848136 podStartE2EDuration="2m28.28092119s" totalImagesPullingTime="2m20.22973859s" totalInitContainerRuntime="4.734334464s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:08:39.277712977 +0000 UTC m=+1088.324908143" watchObservedRunningTime="2026-07-13 19:08:39.28092119 +0000 UTC m=+1088.328116346"
Jul 13 19:08:40.250255 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:40.250213    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:08:40.251445 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:40.251415    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 13 19:08:41.253896 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:41.253857    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 13 19:08:46.259012 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:46.258980    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:08:46.259576 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:46.259457    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 13 19:08:56.260749 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:08:56.260717    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:09:01.829315 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:01.829284    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"]
Jul 13 19:09:01.829816 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:01.829534    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container" containerID="cri-o://6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d" gracePeriod=30
Jul 13 19:09:01.829816 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:01.829588    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kube-rbac-proxy" containerID="cri-o://57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385" gracePeriod=30
Jul 13 19:09:02.353657 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.353626    2581 generic.go:356] "Generic (PLEG): container finished" podID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerID="57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385" exitCode=2
Jul 13 19:09:02.353825 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.353694    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerDied","Data":"57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385"}
Jul 13 19:09:02.743243 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.743212    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:09:02.855465 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn2h9\" (UniqueName: \"kubernetes.io/projected/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-api-access-sn2h9\") pod \"d09d0c06-ecac-4b00-b828-3bd74278d11b\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") "
Jul 13 19:09:02.855957 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855479    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-rbac-proxy-sar-config\") pod \"d09d0c06-ecac-4b00-b828-3bd74278d11b\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") "
Jul 13 19:09:02.855957 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855587    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls\") pod \"d09d0c06-ecac-4b00-b828-3bd74278d11b\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") "
Jul 13 19:09:02.855957 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-cabundle-cert\") pod \"d09d0c06-ecac-4b00-b828-3bd74278d11b\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") "
Jul 13 19:09:02.855957 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d09d0c06-ecac-4b00-b828-3bd74278d11b-kserve-provision-location\") pod \"d09d0c06-ecac-4b00-b828-3bd74278d11b\" (UID: \"d09d0c06-ecac-4b00-b828-3bd74278d11b\") "
Jul 13 19:09:02.855957 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855926    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d09d0c06-ecac-4b00-b828-3bd74278d11b" (UID: "d09d0c06-ecac-4b00-b828-3bd74278d11b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:09:02.856281 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.855983    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d09d0c06-ecac-4b00-b828-3bd74278d11b" (UID: "d09d0c06-ecac-4b00-b828-3bd74278d11b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:09:02.857728 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.857698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d09d0c06-ecac-4b00-b828-3bd74278d11b" (UID: "d09d0c06-ecac-4b00-b828-3bd74278d11b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:09:02.857728 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.857708    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-api-access-sn2h9" (OuterVolumeSpecName: "kube-api-access-sn2h9") pod "d09d0c06-ecac-4b00-b828-3bd74278d11b" (UID: "d09d0c06-ecac-4b00-b828-3bd74278d11b"). InnerVolumeSpecName "kube-api-access-sn2h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:09:02.860473 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.860448    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d09d0c06-ecac-4b00-b828-3bd74278d11b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d09d0c06-ecac-4b00-b828-3bd74278d11b" (UID: "d09d0c06-ecac-4b00-b828-3bd74278d11b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:09:02.957138 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.957058    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d09d0c06-ecac-4b00-b828-3bd74278d11b-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:09:02.957138 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.957086    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn2h9\" (UniqueName: \"kubernetes.io/projected/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-api-access-sn2h9\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:09:02.957138 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.957097    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:09:02.957138 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.957106    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d0c06-ecac-4b00-b828-3bd74278d11b-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:09:02.957138 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:02.957117    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d09d0c06-ecac-4b00-b828-3bd74278d11b-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:09:03.360043 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.359959    2581 generic.go:356] "Generic (PLEG): container finished" podID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerID="6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d" exitCode=0
Jul 13 19:09:03.360043 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.360017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"
Jul 13 19:09:03.360238 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.360042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerDied","Data":"6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d"}
Jul 13 19:09:03.360238 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.360100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb" event={"ID":"d09d0c06-ecac-4b00-b828-3bd74278d11b","Type":"ContainerDied","Data":"faa8f61c223f71e2d9bddf70286c28d29969334b599c04eeac8fac0da0dc7f08"}
Jul 13 19:09:03.360238 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.360130    2581 scope.go:117] "RemoveContainer" containerID="57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385"
Jul 13 19:09:03.368542 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.368528    2581 scope.go:117] "RemoveContainer" containerID="6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d"
Jul 13 19:09:03.375784 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.375765    2581 scope.go:117] "RemoveContainer" containerID="3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592"
Jul 13 19:09:03.382960 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.382944    2581 scope.go:117] "RemoveContainer" containerID="57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385"
Jul 13 19:09:03.383197 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:09:03.383180    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385\": container with ID starting with 57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385 not found: ID does not exist" containerID="57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385"
Jul 13 19:09:03.383235 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.383207    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385"} err="failed to get container status \"57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385\": rpc error: code = NotFound desc = could not find container \"57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385\": container with ID starting with 57a718d1f01bfa596127921aea0c17eb0c16d2840510e7b4dc0c7edecdee7385 not found: ID does not exist"
Jul 13 19:09:03.383235 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.383224    2581 scope.go:117] "RemoveContainer" containerID="6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d"
Jul 13 19:09:03.383449 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:09:03.383420    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d\": container with ID starting with 6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d not found: ID does not exist" containerID="6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d"
Jul 13 19:09:03.383449 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.383438    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d"} err="failed to get container status \"6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d\": rpc error: code = NotFound desc = could not find container \"6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d\": container with ID starting with 6260280831c000aa74f9f48501f18974b007498c1cf8828e9e555c2fb6b8e53d not found: ID does not exist"
Jul 13 19:09:03.383548 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.383455    2581 scope.go:117] "RemoveContainer" containerID="3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592"
Jul 13 19:09:03.383762 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:09:03.383729    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592\": container with ID starting with 3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592 not found: ID does not exist" containerID="3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592"
Jul 13 19:09:03.383834 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.383759    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592"} err="failed to get container status \"3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592\": rpc error: code = NotFound desc = could not find container \"3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592\": container with ID starting with 3ec8c63889d6dffec5f2665b6928e83fe5c657702836c2989fd0f8580882a592 not found: ID does not exist"
Jul 13 19:09:03.386293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.386273    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"]
Jul 13 19:09:03.389090 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.389072    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4chbb"]
Jul 13 19:09:03.506079 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:03.506048    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" path="/var/lib/kubelet/pods/d09d0c06-ecac-4b00-b828-3bd74278d11b/volumes"
Jul 13 19:09:07.167858 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.167822    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"]
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168142    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168164    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168178    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168184    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168193    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kserve-container"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168199    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kserve-container"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168207    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168212    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168218    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="storage-initializer"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168223    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="storage-initializer"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168229    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent"
Jul 13 19:09:07.168239 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168235    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168247    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="storage-initializer"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168252    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="storage-initializer"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168257    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168263    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168274    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168280    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168329    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168341    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kube-rbac-proxy"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168350    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kserve-container"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168358    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="kserve-container"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168365    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56458e18-dcac-4299-8610-9015087e177e" containerName="agent"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168370    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09d0c06-ecac-4b00-b828-3bd74278d11b" containerName="kserve-container"
Jul 13 19:09:07.168615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.168375    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc110fb-2fd1-40d5-b6c7-16462b777aba" containerName="kube-rbac-proxy"
Jul 13 19:09:07.183377 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.183353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.185197 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185140    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"]
Jul 13 19:09:07.185413 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185390    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 13 19:09:07.185523 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185464    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-q9f8j\""
Jul 13 19:09:07.185615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185604    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 13 19:09:07.185733 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185716    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 13 19:09:07.185933 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185919    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 13 19:09:07.186011 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185933    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 13 19:09:07.186011 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.185921    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 13 19:09:07.295034 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.295008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw9p9\" (UniqueName: \"kubernetes.io/projected/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-api-access-bw9p9\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.295226 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.295060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.295226 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.295111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.295226 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.295146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.295226 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.295183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.396214 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.396185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.396214 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.396223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.396517 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.396256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw9p9\" (UniqueName: \"kubernetes.io/projected/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-api-access-bw9p9\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.396517 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.396295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.396517 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.396346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.396704 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.396680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.397102 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.397079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.397143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.397083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.398625 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.398609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.407684 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.407661    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw9p9\" (UniqueName: \"kubernetes.io/projected/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-api-access-bw9p9\") pod \"isvc-autogluon-v2-predictor-67b849d456-xg4r6\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.494856 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.494770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:07.647056 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:07.647031    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"]
Jul 13 19:09:07.649520 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:09:07.649491    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec331ffd_f3b4_4c0e_b422_c8fdaa8fd488.slice/crio-cd0f19b238535d78af30ccfd17e364bbbf99acc9822009749a824fa3daa43fea WatchSource:0}: Error finding container cd0f19b238535d78af30ccfd17e364bbbf99acc9822009749a824fa3daa43fea: Status 404 returned error can't find the container with id cd0f19b238535d78af30ccfd17e364bbbf99acc9822009749a824fa3daa43fea
Jul 13 19:09:08.382980 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:08.382947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerStarted","Data":"4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf"}
Jul 13 19:09:08.383357 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:08.383008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerStarted","Data":"cd0f19b238535d78af30ccfd17e364bbbf99acc9822009749a824fa3daa43fea"}
Jul 13 19:09:38.511675 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:38.511638    2581 generic.go:356] "Generic (PLEG): container finished" podID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerID="4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf" exitCode=0
Jul 13 19:09:38.512066 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:38.511678    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerDied","Data":"4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf"}
Jul 13 19:09:39.518435 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:39.518393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerStarted","Data":"f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc"}
Jul 13 19:09:39.518884 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:39.518478    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerStarted","Data":"dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd"}
Jul 13 19:09:39.518884 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:39.518709    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:39.540547 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:39.540489    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" podStartSLOduration=2.241150994 podStartE2EDuration="32.540469092s" totalImagesPullingTime="0s" totalInitContainerRuntime="30.299318098s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:09:39.538671288 +0000 UTC m=+1148.585866442" watchObservedRunningTime="2026-07-13 19:09:39.540469092 +0000 UTC m=+1148.587664248"
Jul 13 19:09:40.523725 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:40.523692    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:09:46.534709 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:09:46.534680    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:10:31.430314 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:10:31.430283    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:10:31.431834 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:10:31.431802    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:11:13.506634 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:13.506602    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:11:17.488257 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:17.488223    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"]
Jul 13 19:11:17.488831 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:17.488435    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kserve-container" containerID="cri-o://dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd" gracePeriod=30
Jul 13 19:11:17.488831 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:17.488454    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kube-rbac-proxy" containerID="cri-o://f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc" gracePeriod=30
Jul 13 19:11:17.928425 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:17.928391    2581 generic.go:356] "Generic (PLEG): container finished" podID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerID="f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc" exitCode=2
Jul 13 19:11:17.928629 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:17.928461    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerDied","Data":"f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc"}
Jul 13 19:11:18.334233 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.334211    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:11:18.403374 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403346    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-cabundle-cert\") pod \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") "
Jul 13 19:11:18.403374 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403377    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-rbac-proxy-sar-config\") pod \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") "
Jul 13 19:11:18.403632 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403437    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-proxy-tls\") pod \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") "
Jul 13 19:11:18.403632 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403471    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw9p9\" (UniqueName: \"kubernetes.io/projected/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-api-access-bw9p9\") pod \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") "
Jul 13 19:11:18.403632 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kserve-provision-location\") pod \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\" (UID: \"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488\") "
Jul 13 19:11:18.403902 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403859    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" (UID: "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:11:18.403902 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.403868    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" (UID: "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:11:18.405535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.405514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-api-access-bw9p9" (OuterVolumeSpecName: "kube-api-access-bw9p9") pod "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" (UID: "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488"). InnerVolumeSpecName "kube-api-access-bw9p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:11:18.405535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.405529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" (UID: "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:11:18.409836 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.409809    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" (UID: "ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:11:18.504466 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.504386    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:11:18.504466 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.504412    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw9p9\" (UniqueName: \"kubernetes.io/projected/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-api-access-bw9p9\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:11:18.504466 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.504421    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:11:18.504466 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.504430    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:11:18.504466 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.504438    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:11:18.934893 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.934851    2581 generic.go:356] "Generic (PLEG): container finished" podID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerID="dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd" exitCode=0
Jul 13 19:11:18.935070 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.934911    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"
Jul 13 19:11:18.935070 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.934915    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerDied","Data":"dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd"}
Jul 13 19:11:18.935070 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.934976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6" event={"ID":"ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488","Type":"ContainerDied","Data":"cd0f19b238535d78af30ccfd17e364bbbf99acc9822009749a824fa3daa43fea"}
Jul 13 19:11:18.935070 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.935002    2581 scope.go:117] "RemoveContainer" containerID="f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc"
Jul 13 19:11:18.943623 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.943603    2581 scope.go:117] "RemoveContainer" containerID="dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd"
Jul 13 19:11:18.951173 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.951146    2581 scope.go:117] "RemoveContainer" containerID="4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf"
Jul 13 19:11:18.960527 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.960509    2581 scope.go:117] "RemoveContainer" containerID="f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc"
Jul 13 19:11:18.960828 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:11:18.960808    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc\": container with ID starting with f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc not found: ID does not exist" containerID="f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc"
Jul 13 19:11:18.960910 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.960836    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc"} err="failed to get container status \"f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc\": rpc error: code = NotFound desc = could not find container \"f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc\": container with ID starting with f30b7327e9147ba325ff7dd480e0ac53b840c24e6873752264771c10442e40fc not found: ID does not exist"
Jul 13 19:11:18.960910 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.960852    2581 scope.go:117] "RemoveContainer" containerID="dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd"
Jul 13 19:11:18.961088 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.961068    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"]
Jul 13 19:11:18.961142 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:11:18.961094    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd\": container with ID starting with dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd not found: ID does not exist" containerID="dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd"
Jul 13 19:11:18.961142 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.961112    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd"} err="failed to get container status \"dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd\": rpc error: code = NotFound desc = could not find container \"dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd\": container with ID starting with dcf80d1fc3f6ee1a95d371ac6d0b4c18ed68b1122f96159762099c3f815f74fd not found: ID does not exist"
Jul 13 19:11:18.961142 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.961123    2581 scope.go:117] "RemoveContainer" containerID="4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf"
Jul 13 19:11:18.961351 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:11:18.961331    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf\": container with ID starting with 4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf not found: ID does not exist" containerID="4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf"
Jul 13 19:11:18.961409 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.961359    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf"} err="failed to get container status \"4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf\": rpc error: code = NotFound desc = could not find container \"4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf\": container with ID starting with 4302da1b2f6ee7a392ed4ec70e8890464d2522a80c44a18b3a7e187486e128bf not found: ID does not exist"
Jul 13 19:11:18.967329 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:18.967311    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-xg4r6"]
Jul 13 19:11:19.506268 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:19.506238    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" path="/var/lib/kubelet/pods/ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488/volumes"
Jul 13 19:11:22.524636 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524601    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"]
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524923    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kserve-container"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524933    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kserve-container"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524940    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kube-rbac-proxy"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524947    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kube-rbac-proxy"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524956    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="storage-initializer"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.524962    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="storage-initializer"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.525016    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kserve-container"
Jul 13 19:11:22.525143 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.525027    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec331ffd-f3b4-4c0e-b422-c8fdaa8fd488" containerName="kube-rbac-proxy"
Jul 13 19:11:22.528600 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.528579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.531094 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531071    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 13 19:11:22.531233 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531103    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 13 19:11:22.531233 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531075    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 13 19:11:22.531233 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531180    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 13 19:11:22.531631 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531603    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-q9f8j\""
Jul 13 19:11:22.531756 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531645    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 13 19:11:22.531756 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.531603    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 13 19:11:22.536824 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.536418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.536824 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.536487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl57l\" (UniqueName: \"kubernetes.io/projected/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-api-access-gl57l\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.536824 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.536541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.536824 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.536703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.536824 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.536774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.544300 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.544278    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"]
Jul 13 19:11:22.637225 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.637196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.637414 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.637237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl57l\" (UniqueName: \"kubernetes.io/projected/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-api-access-gl57l\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.637414 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.637371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.637538 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.637449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.637538 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.637507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.637873 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.637852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.638167 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.638144    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.638206 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.638152    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.639629 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.639610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.655751 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.655724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl57l\" (UniqueName: \"kubernetes.io/projected/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-api-access-gl57l\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.840179 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.840079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:22.971063 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:22.971040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"]
Jul 13 19:11:22.973304 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:11:22.973275    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639944a3_3d3c_4db7_a8c2_f9ad30e99b38.slice/crio-1aba421d1879237e038ea6d3134bacda82b177df14c5cce20ec25a84ccf15850 WatchSource:0}: Error finding container 1aba421d1879237e038ea6d3134bacda82b177df14c5cce20ec25a84ccf15850: Status 404 returned error can't find the container with id 1aba421d1879237e038ea6d3134bacda82b177df14c5cce20ec25a84ccf15850
Jul 13 19:11:23.958774 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:23.958737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerStarted","Data":"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"}
Jul 13 19:11:23.959200 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:23.958798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerStarted","Data":"1aba421d1879237e038ea6d3134bacda82b177df14c5cce20ec25a84ccf15850"}
Jul 13 19:11:53.080485 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:53.080449    2581 generic.go:356] "Generic (PLEG): container finished" podID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerID="5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69" exitCode=0
Jul 13 19:11:53.080850 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:53.080523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerDied","Data":"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"}
Jul 13 19:11:54.087110 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:54.087077    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerStarted","Data":"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"}
Jul 13 19:11:54.087491 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:54.087137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerStarted","Data":"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"}
Jul 13 19:11:54.087491 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:54.087312    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:11:54.126244 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:54.126192    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" podStartSLOduration=2.16450447 podStartE2EDuration="32.12617598s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.96167151s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:11:54.123786133 +0000 UTC m=+1283.170981288" watchObservedRunningTime="2026-07-13 19:11:54.12617598 +0000 UTC m=+1283.173371134"
Jul 13 19:11:55.091955 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:11:55.091928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:12:01.102671 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:12:01.102644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:13:29.505841 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:29.505814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:13:32.645976 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:32.645944    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"]
Jul 13 19:13:32.646345 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:32.646190    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kserve-container" containerID="cri-o://7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f" gracePeriod=30
Jul 13 19:13:32.646345 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:32.646240    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kube-rbac-proxy" containerID="cri-o://4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec" gracePeriod=30
Jul 13 19:13:33.489801 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.489780    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:13:33.500595 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500571    2581 generic.go:356] "Generic (PLEG): container finished" podID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerID="4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec" exitCode=2
Jul 13 19:13:33.500595 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500590    2581 generic.go:356] "Generic (PLEG): container finished" podID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerID="7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f" exitCode=0
Jul 13 19:13:33.500790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500623    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"
Jul 13 19:13:33.500790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerDied","Data":"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"}
Jul 13 19:13:33.500790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500677    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerDied","Data":"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"}
Jul 13 19:13:33.500790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw" event={"ID":"639944a3-3d3c-4db7-a8c2-f9ad30e99b38","Type":"ContainerDied","Data":"1aba421d1879237e038ea6d3134bacda82b177df14c5cce20ec25a84ccf15850"}
Jul 13 19:13:33.500790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.500723    2581 scope.go:117] "RemoveContainer" containerID="4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"
Jul 13 19:13:33.509570 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.509530    2581 scope.go:117] "RemoveContainer" containerID="7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"
Jul 13 19:13:33.517638 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.517621    2581 scope.go:117] "RemoveContainer" containerID="5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"
Jul 13 19:13:33.527176 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.527092    2581 scope.go:117] "RemoveContainer" containerID="4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"
Jul 13 19:13:33.527640 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:13:33.527615    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec\": container with ID starting with 4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec not found: ID does not exist" containerID="4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"
Jul 13 19:13:33.527732 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.527651    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"} err="failed to get container status \"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec\": rpc error: code = NotFound desc = could not find container \"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec\": container with ID starting with 4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec not found: ID does not exist"
Jul 13 19:13:33.527732 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.527673    2581 scope.go:117] "RemoveContainer" containerID="7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"
Jul 13 19:13:33.527906 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:13:33.527891    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f\": container with ID starting with 7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f not found: ID does not exist" containerID="7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"
Jul 13 19:13:33.527959 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.527909    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"} err="failed to get container status \"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f\": rpc error: code = NotFound desc = could not find container \"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f\": container with ID starting with 7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f not found: ID does not exist"
Jul 13 19:13:33.527959 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.527921    2581 scope.go:117] "RemoveContainer" containerID="5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"
Jul 13 19:13:33.528127 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:13:33.528111    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69\": container with ID starting with 5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69 not found: ID does not exist" containerID="5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"
Jul 13 19:13:33.528182 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528133    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"} err="failed to get container status \"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69\": rpc error: code = NotFound desc = could not find container \"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69\": container with ID starting with 5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69 not found: ID does not exist"
Jul 13 19:13:33.528182 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528145    2581 scope.go:117] "RemoveContainer" containerID="4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"
Jul 13 19:13:33.528370 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528355    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec"} err="failed to get container status \"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec\": rpc error: code = NotFound desc = could not find container \"4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec\": container with ID starting with 4e1d7a97de984057d9ed3da7212a43af14c2e3af4dbf8502c9df3696846c28ec not found: ID does not exist"
Jul 13 19:13:33.528422 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528370    2581 scope.go:117] "RemoveContainer" containerID="7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"
Jul 13 19:13:33.528607 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528587    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f"} err="failed to get container status \"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f\": rpc error: code = NotFound desc = could not find container \"7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f\": container with ID starting with 7c33a65f4c140d4ce723404df18f75a1c9aec98c2f98f49878cd26a56226023f not found: ID does not exist"
Jul 13 19:13:33.528670 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528609    2581 scope.go:117] "RemoveContainer" containerID="5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"
Jul 13 19:13:33.528844 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.528820    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69"} err="failed to get container status \"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69\": rpc error: code = NotFound desc = could not find container \"5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69\": container with ID starting with 5c8172a7b582d98122eea41560c337311c194b02f2447e7ece1acb8e5fcc4e69 not found: ID does not exist"
Jul 13 19:13:33.658949 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.658925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl57l\" (UniqueName: \"kubernetes.io/projected/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-api-access-gl57l\") pod \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") "
Jul 13 19:13:33.659532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.658965    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-rbac-proxy-sar-config\") pod \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") "
Jul 13 19:13:33.659532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.659001    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-cabundle-cert\") pod \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") "
Jul 13 19:13:33.659532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.659036    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-proxy-tls\") pod \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") "
Jul 13 19:13:33.659532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.659055    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kserve-provision-location\") pod \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\" (UID: \"639944a3-3d3c-4db7-a8c2-f9ad30e99b38\") "
Jul 13 19:13:33.659532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.659389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "639944a3-3d3c-4db7-a8c2-f9ad30e99b38" (UID: "639944a3-3d3c-4db7-a8c2-f9ad30e99b38"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:13:33.659532 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.659399    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "639944a3-3d3c-4db7-a8c2-f9ad30e99b38" (UID: "639944a3-3d3c-4db7-a8c2-f9ad30e99b38"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:13:33.661086 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.661060    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "639944a3-3d3c-4db7-a8c2-f9ad30e99b38" (UID: "639944a3-3d3c-4db7-a8c2-f9ad30e99b38"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:13:33.661178 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.661149    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-api-access-gl57l" (OuterVolumeSpecName: "kube-api-access-gl57l") pod "639944a3-3d3c-4db7-a8c2-f9ad30e99b38" (UID: "639944a3-3d3c-4db7-a8c2-f9ad30e99b38"). InnerVolumeSpecName "kube-api-access-gl57l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:13:33.663505 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.663482    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "639944a3-3d3c-4db7-a8c2-f9ad30e99b38" (UID: "639944a3-3d3c-4db7-a8c2-f9ad30e99b38"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:13:33.760415 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.760384    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:13:33.760415 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.760408    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:13:33.760415 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.760419    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl57l\" (UniqueName: \"kubernetes.io/projected/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-api-access-gl57l\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:13:33.760651 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.760428    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:13:33.760651 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.760437    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/639944a3-3d3c-4db7-a8c2-f9ad30e99b38-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:13:33.823271 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.823243    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"]
Jul 13 19:13:33.828501 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:33.828475    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-x7dpw"]
Jul 13 19:13:35.506292 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:35.506256    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" path="/var/lib/kubelet/pods/639944a3-3d3c-4db7-a8c2-f9ad30e99b38/volumes"
Jul 13 19:13:37.824050 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824021    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"]
Jul 13 19:13:37.824417 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824327    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kube-rbac-proxy"
Jul 13 19:13:37.824417 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824339    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kube-rbac-proxy"
Jul 13 19:13:37.824417 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824356    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="storage-initializer"
Jul 13 19:13:37.824417 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824362    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="storage-initializer"
Jul 13 19:13:37.824417 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824371    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kserve-container"
Jul 13 19:13:37.824417 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824376    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kserve-container"
Jul 13 19:13:37.824629 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824428    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kserve-container"
Jul 13 19:13:37.824629 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.824438    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="639944a3-3d3c-4db7-a8c2-f9ad30e99b38" containerName="kube-rbac-proxy"
Jul 13 19:13:37.829114 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.829099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:37.831196 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831174    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 13 19:13:37.831196 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831185    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 13 19:13:37.831432 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 13 19:13:37.831524 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831431    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 13 19:13:37.831524 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831416    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 13 19:13:37.831524 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831455    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 13 19:13:37.831776 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.831759    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-q9f8j\""
Jul 13 19:13:37.837122 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.837101    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"]
Jul 13 19:13:37.997056 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.997023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:37.997233 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.997070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgk7x\" (UniqueName: \"kubernetes.io/projected/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-api-access-cgk7x\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:37.997233 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.997176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:37.997307 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.997227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:37.997307 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:37.997296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.097882 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.097794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.097882 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.097833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.097882 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.097860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgk7x\" (UniqueName: \"kubernetes.io/projected/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-api-access-cgk7x\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.098114 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.097906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.098114 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.097927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.098338 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.098318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.098626 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.098604    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.098675 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.098621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.100280 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.100263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.105775 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.105751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgk7x\" (UniqueName: \"kubernetes.io/projected/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-api-access-cgk7x\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.139719 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.139694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:38.264088 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.264056    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"]
Jul 13 19:13:38.266468 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:13:38.266438    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbe77dd_f47c_4ea0_8429_dc3e3dd2c349.slice/crio-fc70acd5b3e0f6cadfb56863cb1030edd1bf035e44dd19c9cde1fdadafa60276 WatchSource:0}: Error finding container fc70acd5b3e0f6cadfb56863cb1030edd1bf035e44dd19c9cde1fdadafa60276: Status 404 returned error can't find the container with id fc70acd5b3e0f6cadfb56863cb1030edd1bf035e44dd19c9cde1fdadafa60276
Jul 13 19:13:38.521857 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.521819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerStarted","Data":"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"}
Jul 13 19:13:38.522009 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:38.521880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerStarted","Data":"fc70acd5b3e0f6cadfb56863cb1030edd1bf035e44dd19c9cde1fdadafa60276"}
Jul 13 19:13:43.542812 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:43.542776    2581 generic.go:356] "Generic (PLEG): container finished" podID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerID="7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d" exitCode=0
Jul 13 19:13:43.543167 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:43.542852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerDied","Data":"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"}
Jul 13 19:13:44.549081 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:44.549049    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerStarted","Data":"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"}
Jul 13 19:13:44.549478 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:44.549106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerStarted","Data":"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"}
Jul 13 19:13:44.549478 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:44.549377    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:44.569776 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:44.569728    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" podStartSLOduration=2.441519777 podStartE2EDuration="7.569711762s" totalImagesPullingTime="0s" totalInitContainerRuntime="5.128191985s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:13:44.568039606 +0000 UTC m=+1393.615234772" watchObservedRunningTime="2026-07-13 19:13:44.569711762 +0000 UTC m=+1393.616906917"
Jul 13 19:13:45.553739 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:45.553701    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:13:51.563174 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:13:51.563143    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:15:16.504968 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:16.504931    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:15:18.069228 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.069191    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"]
Jul 13 19:15:18.069646 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.069390    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kserve-container" containerID="cri-o://3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6" gracePeriod=30
Jul 13 19:15:18.069646 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.069441    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kube-rbac-proxy" containerID="cri-o://10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730" gracePeriod=30
Jul 13 19:15:18.811850 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.811826    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:15:18.953325 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953244    2581 generic.go:356] "Generic (PLEG): container finished" podID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerID="10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730" exitCode=2
Jul 13 19:15:18.953325 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953266    2581 generic.go:356] "Generic (PLEG): container finished" podID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerID="3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6" exitCode=0
Jul 13 19:15:18.953325 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953286    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerDied","Data":"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"}
Jul 13 19:15:18.953325 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953301    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"
Jul 13 19:15:18.953644 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerDied","Data":"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"}
Jul 13 19:15:18.953644 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn" event={"ID":"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349","Type":"ContainerDied","Data":"fc70acd5b3e0f6cadfb56863cb1030edd1bf035e44dd19c9cde1fdadafa60276"}
Jul 13 19:15:18.953644 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.953387    2581 scope.go:117] "RemoveContainer" containerID="10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"
Jul 13 19:15:18.959955 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.959932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-rbac-proxy-sar-config\") pod \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") "
Jul 13 19:15:18.960095 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.959998    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-cabundle-cert\") pod \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") "
Jul 13 19:15:18.960095 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.960034    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-proxy-tls\") pod \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") "
Jul 13 19:15:18.960095 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.960070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgk7x\" (UniqueName: \"kubernetes.io/projected/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-api-access-cgk7x\") pod \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") "
Jul 13 19:15:18.960388 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.960103    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kserve-provision-location\") pod \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\" (UID: \"adbe77dd-f47c-4ea0-8429-dc3e3dd2c349\") "
Jul 13 19:15:18.960443 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.960398    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" (UID: "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:15:18.960502 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.960468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" (UID: "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:15:18.961850 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.961833    2581 scope.go:117] "RemoveContainer" containerID="3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"
Jul 13 19:15:18.962041 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.962007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-api-access-cgk7x" (OuterVolumeSpecName: "kube-api-access-cgk7x") pod "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" (UID: "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349"). InnerVolumeSpecName "kube-api-access-cgk7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:15:18.962460 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.962432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" (UID: "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:15:18.965345 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.965322    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" (UID: "adbe77dd-f47c-4ea0-8429-dc3e3dd2c349"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:15:18.971577 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.971547    2581 scope.go:117] "RemoveContainer" containerID="7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"
Jul 13 19:15:18.978306 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.978290    2581 scope.go:117] "RemoveContainer" containerID="10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"
Jul 13 19:15:18.978613 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:15:18.978586    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730\": container with ID starting with 10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730 not found: ID does not exist" containerID="10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"
Jul 13 19:15:18.978707 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.978624    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"} err="failed to get container status \"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730\": rpc error: code = NotFound desc = could not find container \"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730\": container with ID starting with 10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730 not found: ID does not exist"
Jul 13 19:15:18.978707 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.978649    2581 scope.go:117] "RemoveContainer" containerID="3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"
Jul 13 19:15:18.978902 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:15:18.978887    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6\": container with ID starting with 3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6 not found: ID does not exist" containerID="3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"
Jul 13 19:15:18.978951 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.978905    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"} err="failed to get container status \"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6\": rpc error: code = NotFound desc = could not find container \"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6\": container with ID starting with 3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6 not found: ID does not exist"
Jul 13 19:15:18.978951 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.978917    2581 scope.go:117] "RemoveContainer" containerID="7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"
Jul 13 19:15:18.979164 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:15:18.979144    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d\": container with ID starting with 7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d not found: ID does not exist" containerID="7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"
Jul 13 19:15:18.979201 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979172    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"} err="failed to get container status \"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d\": rpc error: code = NotFound desc = could not find container \"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d\": container with ID starting with 7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d not found: ID does not exist"
Jul 13 19:15:18.979201 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979191    2581 scope.go:117] "RemoveContainer" containerID="10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"
Jul 13 19:15:18.979430 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979410    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730"} err="failed to get container status \"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730\": rpc error: code = NotFound desc = could not find container \"10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730\": container with ID starting with 10aec5aef623005a4855925ac0ae88034033a2cde794673e52dfe8ce357c8730 not found: ID does not exist"
Jul 13 19:15:18.979471 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979432    2581 scope.go:117] "RemoveContainer" containerID="3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"
Jul 13 19:15:18.979668 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979652    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6"} err="failed to get container status \"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6\": rpc error: code = NotFound desc = could not find container \"3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6\": container with ID starting with 3311521267a9b88d0abf6a1c44c12c4c8c1a17dc21d159b55c0a1ff53338cbd6 not found: ID does not exist"
Jul 13 19:15:18.979732 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979669    2581 scope.go:117] "RemoveContainer" containerID="7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"
Jul 13 19:15:18.979894 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:18.979871    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d"} err="failed to get container status \"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d\": rpc error: code = NotFound desc = could not find container \"7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d\": container with ID starting with 7d4ea9f55627079bea1cdd0e2dc1b40c3eaca2deea817c21f1741df2aa61513d not found: ID does not exist"
Jul 13 19:15:19.061593 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.061537    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:19.061593 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.061589    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:19.061593 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.061601    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:19.061807 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.061609    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgk7x\" (UniqueName: \"kubernetes.io/projected/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kube-api-access-cgk7x\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:19.061807 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.061618    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:19.278685 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.278658    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"]
Jul 13 19:15:19.285599 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.285574    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-nrwcn"]
Jul 13 19:15:19.507699 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:19.507670    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" path="/var/lib/kubelet/pods/adbe77dd-f47c-4ea0-8429-dc3e3dd2c349/volumes"
Jul 13 19:15:23.182642 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182610    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"]
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182937    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="storage-initializer"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182949    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="storage-initializer"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182963    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kube-rbac-proxy"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182970    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kube-rbac-proxy"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182984    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kserve-container"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.182990    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kserve-container"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.183059    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kserve-container"
Jul 13 19:15:23.183103 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.183069    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="adbe77dd-f47c-4ea0-8429-dc3e3dd2c349" containerName="kube-rbac-proxy"
Jul 13 19:15:23.187645 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.187625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.189806 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.189787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 13 19:15:23.190700 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.190685    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 13 19:15:23.190958 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.190934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 13 19:15:23.191546 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.191526    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 13 19:15:23.191636 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.191545    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 13 19:15:23.191865 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.191850    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 13 19:15:23.191990 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.191976    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-q9f8j\""
Jul 13 19:15:23.203047 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.203027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"]
Jul 13 19:15:23.296538 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.296507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4zsk\" (UniqueName: \"kubernetes.io/projected/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-api-access-x4zsk\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.296538 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.296537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.296722 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.296633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf1b0586-6d53-4aa5-9b33-7e35deba1965-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.296722 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.296656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.296722 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.296675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.397861 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.397823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4zsk\" (UniqueName: \"kubernetes.io/projected/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-api-access-x4zsk\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.397861 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.397864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.398057 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.397902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf1b0586-6d53-4aa5-9b33-7e35deba1965-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.398057 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.397918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.398057 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.397934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.398362 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.398341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.398683 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.398663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.398757 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.398675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.400294 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.400276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf1b0586-6d53-4aa5-9b33-7e35deba1965-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.407081 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.407064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4zsk\" (UniqueName: \"kubernetes.io/projected/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-api-access-x4zsk\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.497536 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.497477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:23.627245 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.627219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"]
Jul 13 19:15:23.629782 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:15:23.629758    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf1b0586_6d53_4aa5_9b33_7e35deba1965.slice/crio-5f95af5a0300e4f0fa5dfa93bba18f85cf68ab2b55483b19e5b313626ba244ac WatchSource:0}: Error finding container 5f95af5a0300e4f0fa5dfa93bba18f85cf68ab2b55483b19e5b313626ba244ac: Status 404 returned error can't find the container with id 5f95af5a0300e4f0fa5dfa93bba18f85cf68ab2b55483b19e5b313626ba244ac
Jul 13 19:15:23.976017 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.975980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerStarted","Data":"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"}
Jul 13 19:15:23.976219 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:23.976049    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerStarted","Data":"5f95af5a0300e4f0fa5dfa93bba18f85cf68ab2b55483b19e5b313626ba244ac"}
Jul 13 19:15:31.457785 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:31.457758    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:15:31.459790 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:31.459767    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:15:34.017543 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:34.017507    2581 generic.go:356] "Generic (PLEG): container finished" podID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerID="881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1" exitCode=0
Jul 13 19:15:34.017903 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:34.017589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerDied","Data":"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"}
Jul 13 19:15:35.038531 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:35.038495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerStarted","Data":"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"}
Jul 13 19:15:35.038933 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:35.038572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerStarted","Data":"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"}
Jul 13 19:15:35.038933 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:35.038781    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:35.056890 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:35.056823    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" podStartSLOduration=2.6125253280000003 podStartE2EDuration="12.056809428s" totalImagesPullingTime="0s" totalInitContainerRuntime="9.4442841s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:15:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:15:35.055940971 +0000 UTC m=+1504.103136128" watchObservedRunningTime="2026-07-13 19:15:35.056809428 +0000 UTC m=+1504.104004583"
Jul 13 19:15:36.043419 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:36.043385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:36.044600 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:36.044574    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 13 19:15:37.047995 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:37.047947    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 13 19:15:42.053529 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:42.053499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:42.054145 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:42.054117    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 13 19:15:52.055245 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:52.055212    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:53.214106 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:53.214037    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"]
Jul 13 19:15:53.214469 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:53.214251    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container" containerID="cri-o://1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358" gracePeriod=30
Jul 13 19:15:53.214469 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:53.214316    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kube-rbac-proxy" containerID="cri-o://818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa" gracePeriod=30
Jul 13 19:15:53.955621 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:53.955599    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:54.063741 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.063677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-cabundle-cert\") pod \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") "
Jul 13 19:15:54.063978 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.063750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4zsk\" (UniqueName: \"kubernetes.io/projected/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-api-access-x4zsk\") pod \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") "
Jul 13 19:15:54.063978 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.063889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kserve-provision-location\") pod \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") "
Jul 13 19:15:54.063978 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.063937    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-rbac-proxy-sar-config\") pod \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") "
Jul 13 19:15:54.064146 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.063998    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf1b0586-6d53-4aa5-9b33-7e35deba1965-proxy-tls\") pod \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\" (UID: \"bf1b0586-6d53-4aa5-9b33-7e35deba1965\") "
Jul 13 19:15:54.064146 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.064013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "bf1b0586-6d53-4aa5-9b33-7e35deba1965" (UID: "bf1b0586-6d53-4aa5-9b33-7e35deba1965"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:15:54.064271 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.064245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bf1b0586-6d53-4aa5-9b33-7e35deba1965" (UID: "bf1b0586-6d53-4aa5-9b33-7e35deba1965"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:15:54.064334 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.064319    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-cabundle-cert\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:54.064400 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.064338    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:54.064872 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.064853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "bf1b0586-6d53-4aa5-9b33-7e35deba1965" (UID: "bf1b0586-6d53-4aa5-9b33-7e35deba1965"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:15:54.065798 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.065776    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-api-access-x4zsk" (OuterVolumeSpecName: "kube-api-access-x4zsk") pod "bf1b0586-6d53-4aa5-9b33-7e35deba1965" (UID: "bf1b0586-6d53-4aa5-9b33-7e35deba1965"). InnerVolumeSpecName "kube-api-access-x4zsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:15:54.065903 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.065887    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf1b0586-6d53-4aa5-9b33-7e35deba1965-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bf1b0586-6d53-4aa5-9b33-7e35deba1965" (UID: "bf1b0586-6d53-4aa5-9b33-7e35deba1965"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:15:54.123746 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123715    2581 generic.go:356] "Generic (PLEG): container finished" podID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerID="818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa" exitCode=2
Jul 13 19:15:54.123746 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123741    2581 generic.go:356] "Generic (PLEG): container finished" podID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerID="1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358" exitCode=0
Jul 13 19:15:54.123891 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123765    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"
Jul 13 19:15:54.123891 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerDied","Data":"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"}
Jul 13 19:15:54.123891 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerDied","Data":"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"}
Jul 13 19:15:54.123891 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt" event={"ID":"bf1b0586-6d53-4aa5-9b33-7e35deba1965","Type":"ContainerDied","Data":"5f95af5a0300e4f0fa5dfa93bba18f85cf68ab2b55483b19e5b313626ba244ac"}
Jul 13 19:15:54.124018 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.123910    2581 scope.go:117] "RemoveContainer" containerID="818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"
Jul 13 19:15:54.132217 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.132197    2581 scope.go:117] "RemoveContainer" containerID="1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"
Jul 13 19:15:54.139330 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.139315    2581 scope.go:117] "RemoveContainer" containerID="881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"
Jul 13 19:15:54.146710 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.146531    2581 scope.go:117] "RemoveContainer" containerID="818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"
Jul 13 19:15:54.146950 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:15:54.146933    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa\": container with ID starting with 818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa not found: ID does not exist" containerID="818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"
Jul 13 19:15:54.147025 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.146960    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"} err="failed to get container status \"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa\": rpc error: code = NotFound desc = could not find container \"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa\": container with ID starting with 818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa not found: ID does not exist"
Jul 13 19:15:54.147025 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.146977    2581 scope.go:117] "RemoveContainer" containerID="1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"
Jul 13 19:15:54.147201 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:15:54.147182    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358\": container with ID starting with 1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358 not found: ID does not exist" containerID="1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"
Jul 13 19:15:54.147246 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.147207    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"} err="failed to get container status \"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358\": rpc error: code = NotFound desc = could not find container \"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358\": container with ID starting with 1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358 not found: ID does not exist"
Jul 13 19:15:54.147246 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.147225    2581 scope.go:117] "RemoveContainer" containerID="881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"
Jul 13 19:15:54.147466 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:15:54.147448    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1\": container with ID starting with 881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1 not found: ID does not exist" containerID="881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"
Jul 13 19:15:54.147533 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.147484    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"} err="failed to get container status \"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1\": rpc error: code = NotFound desc = could not find container \"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1\": container with ID starting with 881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1 not found: ID does not exist"
Jul 13 19:15:54.147533 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.147505    2581 scope.go:117] "RemoveContainer" containerID="818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"
Jul 13 19:15:54.147800 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.147769    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa"} err="failed to get container status \"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa\": rpc error: code = NotFound desc = could not find container \"818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa\": container with ID starting with 818e997280f009db99b9271df9b2a0baa64f3985c859b5712436c06945c26cfa not found: ID does not exist"
Jul 13 19:15:54.147854 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.147803    2581 scope.go:117] "RemoveContainer" containerID="1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"
Jul 13 19:15:54.148044 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.148027    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358"} err="failed to get container status \"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358\": rpc error: code = NotFound desc = could not find container \"1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358\": container with ID starting with 1ee315e4eeae1a237b13972dc555668fee076445c6ba7d2aff10b4945360a358 not found: ID does not exist"
Jul 13 19:15:54.148099 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.148046    2581 scope.go:117] "RemoveContainer" containerID="881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"
Jul 13 19:15:54.148218 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.148201    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"]
Jul 13 19:15:54.148275 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.148257    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1"} err="failed to get container status \"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1\": rpc error: code = NotFound desc = could not find container \"881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1\": container with ID starting with 881a43dcd12ade384004e43c257b3aa51ba70ad2a7f3544133ad6334d14c4aa1 not found: ID does not exist"
Jul 13 19:15:54.151613 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.151593    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-s27vt"]
Jul 13 19:15:54.165698 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.165678    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4zsk\" (UniqueName: \"kubernetes.io/projected/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kube-api-access-x4zsk\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:54.165698 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.165698    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf1b0586-6d53-4aa5-9b33-7e35deba1965-kserve-provision-location\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:54.165821 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:54.165709    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf1b0586-6d53-4aa5-9b33-7e35deba1965-proxy-tls\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:15:55.506488 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:55.506453    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" path="/var/lib/kubelet/pods/bf1b0586-6d53-4aa5-9b33-7e35deba1965/volumes"
Jul 13 19:15:59.543023 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.542995    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vjqzd/must-gather-qglmk"]
Jul 13 19:15:59.543487 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543467    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container"
Jul 13 19:15:59.543588 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543491    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container"
Jul 13 19:15:59.543588 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543504    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="storage-initializer"
Jul 13 19:15:59.543588 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543513    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="storage-initializer"
Jul 13 19:15:59.543588 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543531    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kube-rbac-proxy"
Jul 13 19:15:59.543588 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543542    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kube-rbac-proxy"
Jul 13 19:15:59.543850 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543651    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kube-rbac-proxy"
Jul 13 19:15:59.543850 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.543667    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1b0586-6d53-4aa5-9b33-7e35deba1965" containerName="kserve-container"
Jul 13 19:15:59.548479 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.548458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.550545 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.550525    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vjqzd\"/\"kube-root-ca.crt\""
Jul 13 19:15:59.550648 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.550549    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vjqzd\"/\"openshift-service-ca.crt\""
Jul 13 19:15:59.550648 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.550590    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vjqzd\"/\"default-dockercfg-xvcb7\""
Jul 13 19:15:59.563547 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.563520    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vjqzd/must-gather-qglmk"]
Jul 13 19:15:59.709793 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.709767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ck5\" (UniqueName: \"kubernetes.io/projected/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-kube-api-access-57ck5\") pod \"must-gather-qglmk\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") " pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.709959 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.709828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-must-gather-output\") pod \"must-gather-qglmk\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") " pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.811219 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.811142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-must-gather-output\") pod \"must-gather-qglmk\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") " pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.811219 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.811210    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ck5\" (UniqueName: \"kubernetes.io/projected/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-kube-api-access-57ck5\") pod \"must-gather-qglmk\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") " pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.811486 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.811466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-must-gather-output\") pod \"must-gather-qglmk\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") " pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.819988 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.819953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ck5\" (UniqueName: \"kubernetes.io/projected/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-kube-api-access-57ck5\") pod \"must-gather-qglmk\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") " pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:15:59.879394 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:15:59.879372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:00.005230 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:00.005208    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vjqzd/must-gather-qglmk"]
Jul 13 19:16:00.007426 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:16:00.007395    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25ee3790_6f75_4f1d_a3f2_50f9c1c231c0.slice/crio-8eb8c0b692b8f8f430e56f72f2cea2c97116f5890a3a9e150e96ddbf6f7fd3a4 WatchSource:0}: Error finding container 8eb8c0b692b8f8f430e56f72f2cea2c97116f5890a3a9e150e96ddbf6f7fd3a4: Status 404 returned error can't find the container with id 8eb8c0b692b8f8f430e56f72f2cea2c97116f5890a3a9e150e96ddbf6f7fd3a4
Jul 13 19:16:00.009279 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:00.009259    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:16:00.149816 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:00.149792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vjqzd/must-gather-qglmk" event={"ID":"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0","Type":"ContainerStarted","Data":"8eb8c0b692b8f8f430e56f72f2cea2c97116f5890a3a9e150e96ddbf6f7fd3a4"}
Jul 13 19:16:07.192504 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:07.192465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vjqzd/must-gather-qglmk" event={"ID":"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0","Type":"ContainerStarted","Data":"f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759"}
Jul 13 19:16:07.192980 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:07.192514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vjqzd/must-gather-qglmk" event={"ID":"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0","Type":"ContainerStarted","Data":"190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"}
Jul 13 19:16:07.208983 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:07.208939    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vjqzd/must-gather-qglmk" podStartSLOduration=1.9512790880000002 podStartE2EDuration="8.208923734s" totalImagesPullingTime="6.257644646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:16:07.207616948 +0000 UTC m=+1536.254812105" watchObservedRunningTime="2026-07-13 19:16:07.208923734 +0000 UTC m=+1536.256118888"
Jul 13 19:16:24.277054 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.277020    2581 generic.go:356] "Generic (PLEG): container finished" podID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerID="190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77" exitCode=0
Jul 13 19:16:24.277465 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.277093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vjqzd/must-gather-qglmk" event={"ID":"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0","Type":"ContainerDied","Data":"190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"}
Jul 13 19:16:24.277465 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.277369    2581 scope.go:117] "RemoveContainer" containerID="190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"
Jul 13 19:16:24.413514 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.413463    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vjqzd_must-gather-qglmk_25ee3790-6f75-4f1d-a3f2-50f9c1c231c0/gather/0.log"
Jul 13 19:16:24.957723 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.957686    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nvxbd/must-gather-cq9fl"]
Jul 13 19:16:24.960247 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.960232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:24.962802 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.962772    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nvxbd\"/\"openshift-service-ca.crt\""
Jul 13 19:16:24.962802 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.962797    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nvxbd\"/\"kube-root-ca.crt\""
Jul 13 19:16:24.962986 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.962820    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nvxbd\"/\"default-dockercfg-mt96z\""
Jul 13 19:16:24.972224 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:24.972206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nvxbd/must-gather-cq9fl"]
Jul 13 19:16:25.024191 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.024167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/391f45e8-f401-4381-806f-7a990a7620ed-must-gather-output\") pod \"must-gather-cq9fl\" (UID: \"391f45e8-f401-4381-806f-7a990a7620ed\") " pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.024319 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.024235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-697xq\" (UniqueName: \"kubernetes.io/projected/391f45e8-f401-4381-806f-7a990a7620ed-kube-api-access-697xq\") pod \"must-gather-cq9fl\" (UID: \"391f45e8-f401-4381-806f-7a990a7620ed\") " pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.124952 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.124916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-697xq\" (UniqueName: \"kubernetes.io/projected/391f45e8-f401-4381-806f-7a990a7620ed-kube-api-access-697xq\") pod \"must-gather-cq9fl\" (UID: \"391f45e8-f401-4381-806f-7a990a7620ed\") " pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.125140 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.124967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/391f45e8-f401-4381-806f-7a990a7620ed-must-gather-output\") pod \"must-gather-cq9fl\" (UID: \"391f45e8-f401-4381-806f-7a990a7620ed\") " pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.125285 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.125267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/391f45e8-f401-4381-806f-7a990a7620ed-must-gather-output\") pod \"must-gather-cq9fl\" (UID: \"391f45e8-f401-4381-806f-7a990a7620ed\") " pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.136969 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.136942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-697xq\" (UniqueName: \"kubernetes.io/projected/391f45e8-f401-4381-806f-7a990a7620ed-kube-api-access-697xq\") pod \"must-gather-cq9fl\" (UID: \"391f45e8-f401-4381-806f-7a990a7620ed\") " pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.269091 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.269018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nvxbd/must-gather-cq9fl"
Jul 13 19:16:25.398781 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:25.398720    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nvxbd/must-gather-cq9fl"]
Jul 13 19:16:25.401489 ip-10-0-134-48 kubenswrapper[2581]: W0713 19:16:25.401463    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod391f45e8_f401_4381_806f_7a990a7620ed.slice/crio-959b9e88ac27fb6450491d8cddaa22686ff31ce06b51da5cd2e27823c511b40a WatchSource:0}: Error finding container 959b9e88ac27fb6450491d8cddaa22686ff31ce06b51da5cd2e27823c511b40a: Status 404 returned error can't find the container with id 959b9e88ac27fb6450491d8cddaa22686ff31ce06b51da5cd2e27823c511b40a
Jul 13 19:16:26.287996 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:26.287965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvxbd/must-gather-cq9fl" event={"ID":"391f45e8-f401-4381-806f-7a990a7620ed","Type":"ContainerStarted","Data":"bac05e8fe6d77d661c0fa3eec37d7a865064f208a6a02750c9472858fb324b67"}
Jul 13 19:16:26.288130 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:26.288009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvxbd/must-gather-cq9fl" event={"ID":"391f45e8-f401-4381-806f-7a990a7620ed","Type":"ContainerStarted","Data":"959b9e88ac27fb6450491d8cddaa22686ff31ce06b51da5cd2e27823c511b40a"}
Jul 13 19:16:27.295984 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:27.295946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvxbd/must-gather-cq9fl" event={"ID":"391f45e8-f401-4381-806f-7a990a7620ed","Type":"ContainerStarted","Data":"a84be43b62fcdfdf59d80fcee78d6e851eb4486f6833377faa852f541a90211c"}
Jul 13 19:16:27.312190 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:27.312141    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nvxbd/must-gather-cq9fl" podStartSLOduration=2.599827458 podStartE2EDuration="3.312126021s" totalImagesPullingTime="712.298563ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:16:27.310839529 +0000 UTC m=+1556.358034685" watchObservedRunningTime="2026-07-13 19:16:27.312126021 +0000 UTC m=+1556.359321176"
Jul 13 19:16:27.600380 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:27.600310    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lvmn2_27995e44-0dd7-4291-91d5-d0823bd49fb5/global-pull-secret-syncer/0.log"
Jul 13 19:16:27.708240 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:27.708212    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t4pf9_cb3ba19f-0cbf-4239-b5f2-7cb95490561a/konnectivity-agent/0.log"
Jul 13 19:16:27.755072 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:27.755041    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-48.ec2.internal_648a1691dcc256654d0f1487adfe8cf6/haproxy/0.log"
Jul 13 19:16:29.792998 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:29.792965    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-vjqzd/must-gather-qglmk"]
Jul 13 19:16:29.793943 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:29.793907    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-vjqzd/must-gather-qglmk" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="copy" containerID="cri-o://f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759" gracePeriod=2
Jul 13 19:16:29.795529 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:29.795418    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qglmk\" is forbidden: User \"system:node:ip-10-0-134-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vjqzd\": no relationship found between node 'ip-10-0-134-48.ec2.internal' and this object" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:29.797671 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:29.797640    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-vjqzd/must-gather-qglmk"]
Jul 13 19:16:30.195936 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.194662    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vjqzd_must-gather-qglmk_25ee3790-6f75-4f1d-a3f2-50f9c1c231c0/copy/0.log"
Jul 13 19:16:30.197686 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.197662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:30.201765 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:30.201730    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qglmk\" is forbidden: User \"system:node:ip-10-0-134-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vjqzd\": no relationship found between node 'ip-10-0-134-48.ec2.internal' and this object" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:30.280981 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.280938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57ck5\" (UniqueName: \"kubernetes.io/projected/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-kube-api-access-57ck5\") pod \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") "
Jul 13 19:16:30.281180 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.281014    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-must-gather-output\") pod \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\" (UID: \"25ee3790-6f75-4f1d-a3f2-50f9c1c231c0\") "
Jul 13 19:16:30.282536 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.282497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" (UID: "25ee3790-6f75-4f1d-a3f2-50f9c1c231c0"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:16:30.286447 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.286295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-kube-api-access-57ck5" (OuterVolumeSpecName: "kube-api-access-57ck5") pod "25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" (UID: "25ee3790-6f75-4f1d-a3f2-50f9c1c231c0"). InnerVolumeSpecName "kube-api-access-57ck5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:16:30.317380 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.317299    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vjqzd_must-gather-qglmk_25ee3790-6f75-4f1d-a3f2-50f9c1c231c0/copy/0.log"
Jul 13 19:16:30.318014 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.317989    2581 generic.go:356] "Generic (PLEG): container finished" podID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerID="f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759" exitCode=143
Jul 13 19:16:30.318615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.318186    2581 scope.go:117] "RemoveContainer" containerID="f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759"
Jul 13 19:16:30.318615 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.318368    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:30.321789 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:30.321761    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qglmk\" is forbidden: User \"system:node:ip-10-0-134-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vjqzd\": no relationship found between node 'ip-10-0-134-48.ec2.internal' and this object" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:30.332626 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.332356    2581 scope.go:117] "RemoveContainer" containerID="190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"
Jul 13 19:16:30.356423 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.356320    2581 scope.go:117] "RemoveContainer" containerID="f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759"
Jul 13 19:16:30.356867 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:30.356829    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759\": container with ID starting with f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759 not found: ID does not exist" containerID="f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759"
Jul 13 19:16:30.357048 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.357017    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759"} err="failed to get container status \"f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759\": rpc error: code = NotFound desc = could not find container \"f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759\": container with ID starting with f49dd749b3b5c4502d529ff5ccdcc264dd66034ef5eb04bb4a827755fb9cb759 not found: ID does not exist"
Jul 13 19:16:30.357175 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.357159    2581 scope.go:117] "RemoveContainer" containerID="190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"
Jul 13 19:16:30.357615 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:30.357582    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77\": container with ID starting with 190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77 not found: ID does not exist" containerID="190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"
Jul 13 19:16:30.357719 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.357620    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77"} err="failed to get container status \"190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77\": rpc error: code = NotFound desc = could not find container \"190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77\": container with ID starting with 190f6f6c6042d1c9a252c5263ae4c43e71ce202d5e5604378f3aec14282bcd77 not found: ID does not exist"
Jul 13 19:16:30.362530 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:30.362504    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qglmk\" is forbidden: User \"system:node:ip-10-0-134-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vjqzd\": no relationship found between node 'ip-10-0-134-48.ec2.internal' and this object" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:30.382295 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.382268    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57ck5\" (UniqueName: \"kubernetes.io/projected/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-kube-api-access-57ck5\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:16:30.382485 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:30.382468    2581 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0-must-gather-output\") on node \"ip-10-0-134-48.ec2.internal\" DevicePath \"\""
Jul 13 19:16:31.509914 ip-10-0-134-48 kubenswrapper[2581]: E0713 19:16:31.509877    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qglmk\" is forbidden: User \"system:node:ip-10-0-134-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vjqzd\": no relationship found between node 'ip-10-0-134-48.ec2.internal' and this object" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" pod="openshift-must-gather-vjqzd/must-gather-qglmk"
Jul 13 19:16:31.516671 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:31.516138    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" path="/var/lib/kubelet/pods/25ee3790-6f75-4f1d-a3f2-50f9c1c231c0/volumes"
Jul 13 19:16:31.961770 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:31.961738    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/alertmanager/0.log"
Jul 13 19:16:31.990523 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:31.990494    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/config-reloader/0.log"
Jul 13 19:16:32.016285 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.016250    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/kube-rbac-proxy-web/0.log"
Jul 13 19:16:32.048385 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.048313    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/kube-rbac-proxy/0.log"
Jul 13 19:16:32.077576 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.077533    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/kube-rbac-proxy-metric/0.log"
Jul 13 19:16:32.103530 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.103500    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/prom-label-proxy/0.log"
Jul 13 19:16:32.129706 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.129678    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af340ad3-9c0f-4ec5-8318-1cb719fa89ec/init-config-reloader/0.log"
Jul 13 19:16:32.195586 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.195525    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nz8bs_f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6/kube-state-metrics/0.log"
Jul 13 19:16:32.218382 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.218333    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nz8bs_f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6/kube-rbac-proxy-main/0.log"
Jul 13 19:16:32.249394 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.249348    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nz8bs_f7f788a5-4fe2-4e47-81a1-1ccf6bd484c6/kube-rbac-proxy-self/0.log"
Jul 13 19:16:32.277792 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.277765    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c55bf4bb9-n69qb_45ba1367-adf0-4897-a4b5-80c22d4f60e9/metrics-server/0.log"
Jul 13 19:16:32.313535 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.313428    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-wwfb8_f95b438c-9ea3-43ce-b2b8-3b7631a3731b/monitoring-plugin/0.log"
Jul 13 19:16:32.524437 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.524408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-npsqx_7da60cc6-7300-4da3-9c10-dba51771c6dd/node-exporter/0.log"
Jul 13 19:16:32.558293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.558264    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-npsqx_7da60cc6-7300-4da3-9c10-dba51771c6dd/kube-rbac-proxy/0.log"
Jul 13 19:16:32.581522 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.581435    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-npsqx_7da60cc6-7300-4da3-9c10-dba51771c6dd/init-textfile/0.log"
Jul 13 19:16:32.610677 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.610648    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-csgmg_9654d578-32ab-4a63-b69c-675cd9c1a38e/kube-rbac-proxy-main/0.log"
Jul 13 19:16:32.633977 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.633943    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-csgmg_9654d578-32ab-4a63-b69c-675cd9c1a38e/kube-rbac-proxy-self/0.log"
Jul 13 19:16:32.661657 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.661526    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-csgmg_9654d578-32ab-4a63-b69c-675cd9c1a38e/openshift-state-metrics/0.log"
Jul 13 19:16:32.703974 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.703893    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/prometheus/0.log"
Jul 13 19:16:32.726968 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.726936    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/config-reloader/0.log"
Jul 13 19:16:32.751358 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.751313    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/thanos-sidecar/0.log"
Jul 13 19:16:32.781044 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.781004    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/kube-rbac-proxy-web/0.log"
Jul 13 19:16:32.805868 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.805840    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/kube-rbac-proxy/0.log"
Jul 13 19:16:32.831456 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.831429    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/kube-rbac-proxy-thanos/0.log"
Jul 13 19:16:32.856943 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.856863    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_07268472-894b-4c51-b4d4-955217a17a41/init-config-reloader/0.log"
Jul 13 19:16:32.893223 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.893195    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-b9xg5_d00969cc-c4a3-444f-bad3-2cfe468feed2/prometheus-operator/0.log"
Jul 13 19:16:32.915531 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.915502    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-b9xg5_d00969cc-c4a3-444f-bad3-2cfe468feed2/kube-rbac-proxy/0.log"
Jul 13 19:16:32.943832 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:32.943800    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-n8gs4_c7ab8b22-9042-40b9-bcd7-8bb1e134c8a7/prometheus-operator-admission-webhook/0.log"
Jul 13 19:16:33.097105 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:33.097073    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b594cd9d-2vcp6_8867943a-2d0d-46ca-8551-61ccc60d5999/thanos-query/0.log"
Jul 13 19:16:33.127761 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:33.127677    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b594cd9d-2vcp6_8867943a-2d0d-46ca-8551-61ccc60d5999/kube-rbac-proxy-web/0.log"
Jul 13 19:16:33.154546 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:33.154515    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b594cd9d-2vcp6_8867943a-2d0d-46ca-8551-61ccc60d5999/kube-rbac-proxy/0.log"
Jul 13 19:16:33.183489 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:33.183458    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b594cd9d-2vcp6_8867943a-2d0d-46ca-8551-61ccc60d5999/prom-label-proxy/0.log"
Jul 13 19:16:33.220749 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:33.220718    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b594cd9d-2vcp6_8867943a-2d0d-46ca-8551-61ccc60d5999/kube-rbac-proxy-rules/0.log"
Jul 13 19:16:33.265625 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:33.265538    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b594cd9d-2vcp6_8867943a-2d0d-46ca-8551-61ccc60d5999/kube-rbac-proxy-metrics/0.log"
Jul 13 19:16:36.070084 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070048    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"]
Jul 13 19:16:36.070656 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070388    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="copy"
Jul 13 19:16:36.070656 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070400    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="copy"
Jul 13 19:16:36.070656 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070416    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="gather"
Jul 13 19:16:36.070656 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070423    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="gather"
Jul 13 19:16:36.070656 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070488    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="gather"
Jul 13 19:16:36.070656 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.070498    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ee3790-6f75-4f1d-a3f2-50f9c1c231c0" containerName="copy"
Jul 13 19:16:36.076241 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.076212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.083766 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.083740    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"]
Jul 13 19:16:36.138545 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.138514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-podres\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.138545 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.138565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-lib-modules\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.138819 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.138588    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxfc5\" (UniqueName: \"kubernetes.io/projected/e1251ff4-1b21-4917-8330-fc7f047d76ff-kube-api-access-sxfc5\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.138819 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.138706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-sys\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.138819 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.138778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-proc\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240209 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-sys\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240390 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-proc\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240390 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-podres\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240390 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-sys\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240390 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-lib-modules\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240390 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxfc5\" (UniqueName: \"kubernetes.io/projected/e1251ff4-1b21-4917-8330-fc7f047d76ff-kube-api-access-sxfc5\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240623 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-proc\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240623 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240434    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-lib-modules\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.240623 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.240487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e1251ff4-1b21-4917-8330-fc7f047d76ff-podres\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.249293 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.248511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxfc5\" (UniqueName: \"kubernetes.io/projected/e1251ff4-1b21-4917-8330-fc7f047d76ff-kube-api-access-sxfc5\") pod \"perf-node-gather-daemonset-l6pb4\" (UID: \"e1251ff4-1b21-4917-8330-fc7f047d76ff\") " pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.389475 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.389440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:36.562993 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:36.562959    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"]
Jul 13 19:16:37.033968 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.033938    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7v5n_6083f9c2-e560-42fb-a5c5-52cb000f7920/dns/0.log"
Jul 13 19:16:37.056277 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.056248    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7v5n_6083f9c2-e560-42fb-a5c5-52cb000f7920/kube-rbac-proxy/0.log"
Jul 13 19:16:37.178801 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.178777    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r4459_d4f1d1b0-33f6-4248-b57e-e9af5bd4af07/dns-node-resolver/0.log"
Jul 13 19:16:37.397333 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.397290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4" event={"ID":"e1251ff4-1b21-4917-8330-fc7f047d76ff","Type":"ContainerStarted","Data":"48c4cf3c3869522f6b5a5243ed74b06782d1d9a4054808efd0f57042018aa6da"}
Jul 13 19:16:37.397523 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.397363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4" event={"ID":"e1251ff4-1b21-4917-8330-fc7f047d76ff","Type":"ContainerStarted","Data":"5dcf49a7d64252acf5c1b61187324cc9d1bfb9030138380630808bfacf9c8b7d"}
Jul 13 19:16:37.398065 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.398037    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:37.414664 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.414606    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4" podStartSLOduration=1.414591902 podStartE2EDuration="1.414591902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:16:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:16:37.413659377 +0000 UTC m=+1566.460854524" watchObservedRunningTime="2026-07-13 19:16:37.414591902 +0000 UTC m=+1566.461787057"
Jul 13 19:16:37.723137 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:37.723060    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97w5n_e291613f-af89-4da4-827c-aaf0c2819787/node-ca/0.log"
Jul 13 19:16:38.977858 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:38.977807    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9t4gz_67d203d3-7769-4e27-b2f8-b928e9094e72/serve-healthcheck-canary/0.log"
Jul 13 19:16:39.443069 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:39.443020    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkttc_8c0c6a97-5c16-4552-8260-e3b3eaa173e1/kube-rbac-proxy/0.log"
Jul 13 19:16:39.467073 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:39.467047    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkttc_8c0c6a97-5c16-4552-8260-e3b3eaa173e1/exporter/0.log"
Jul 13 19:16:39.489316 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:39.489296    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkttc_8c0c6a97-5c16-4552-8260-e3b3eaa173e1/extractor/0.log"
Jul 13 19:16:41.730082 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.730049    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-wr82s_b0a9be78-5dfc-4193-b031-b46af25fc6a6/s3-init/0.log"
Jul 13 19:16:41.751186 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.751161    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-wr82s_b0a9be78-5dfc-4193-b031-b46af25fc6a6/download-hf-model/0.log"
Jul 13 19:16:41.782395 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.782351    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-tggj9_8929db73-95e4-4b09-b70f-52a6929c51c9/s3-tls-init-custom/0.log"
Jul 13 19:16:41.798661 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.798639    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-tggj9_8929db73-95e4-4b09-b70f-52a6929c51c9/download-hf-model/0.log"
Jul 13 19:16:41.832832 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.832808    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-zq9zh_ffba2a3f-aaab-4f99-97ad-d358e9089955/s3-tls-init-serving/0.log"
Jul 13 19:16:41.849571 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.849532    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-zq9zh_ffba2a3f-aaab-4f99-97ad-d358e9089955/download-hf-model/0.log"
Jul 13 19:16:41.899583 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.899543    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-qkf7z_13fdfea4-f476-45d5-aa29-5b935bd5f196/seaweedfs-tls-custom/0.log"
Jul 13 19:16:41.922774 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:41.922750    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-serving-67b7477c6d-6t5vp_432accba-c370-4e93-baea-274a9cc55963/seaweedfs-tls-serving/0.log"
Jul 13 19:16:44.417111 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:44.417080    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nvxbd/perf-node-gather-daemonset-l6pb4"
Jul 13 19:16:47.634009 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.633983    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/kube-multus-additional-cni-plugins/0.log"
Jul 13 19:16:47.656393 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.656369    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/egress-router-binary-copy/0.log"
Jul 13 19:16:47.677920 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.677892    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/cni-plugins/0.log"
Jul 13 19:16:47.699531 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.699510    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/bond-cni-plugin/0.log"
Jul 13 19:16:47.721132 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.721109    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/routeoverride-cni/0.log"
Jul 13 19:16:47.742781 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.742752    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/whereabouts-cni-bincopy/0.log"
Jul 13 19:16:47.763428 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.763403    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zcdgx_38fbac4d-e67e-47ae-8dd7-ed5b9c9f033e/whereabouts-cni/0.log"
Jul 13 19:16:47.797161 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.797142    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-prhls_a9ba7735-1318-4b8f-955e-4f8eeb6eb9f1/kube-multus/0.log"
Jul 13 19:16:47.873018 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.872987    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5xfpm_da08ff36-fa0f-4526-a23e-e2c759749ce6/network-metrics-daemon/0.log"
Jul 13 19:16:47.893881 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:47.893814    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5xfpm_da08ff36-fa0f-4526-a23e-e2c759749ce6/kube-rbac-proxy/0.log"
Jul 13 19:16:48.775492 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.775459    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-controller/0.log"
Jul 13 19:16:48.793704 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.793674    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/1.log"
Jul 13 19:16:48.793829 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.793712    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovn-acl-logging/0.log"
Jul 13 19:16:48.816974 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.816956    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/kube-rbac-proxy-node/0.log"
Jul 13 19:16:48.840398 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.840340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/kube-rbac-proxy-ovn-metrics/0.log"
Jul 13 19:16:48.860546 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.860524    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/northd/0.log"
Jul 13 19:16:48.881521 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.881495    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/nbdb/0.log"
Jul 13 19:16:48.903659 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.903637    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/sbdb/0.log"
Jul 13 19:16:48.948730 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:48.948702    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-frplb_648f005d-b22e-4abd-a731-89b710f8d991/ovnkube-controller/0.log"
Jul 13 19:16:50.359040 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:50.359008    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-77cd8_a79097ef-170e-4a96-9005-914625cb592b/network-check-target-container/0.log"
Jul 13 19:16:51.300534 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:51.300505    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-6bwhr_d71dc73c-f80c-4746-887c-4b0d18073d53/iptables-alerter/0.log"
Jul 13 19:16:51.994610 ip-10-0-134-48 kubenswrapper[2581]: I0713 19:16:51.994579    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4dq2k_2227d0e7-3b39-46be-8b94-fdc23bd64eac/tuned/0.log"