Jul 01 19:21:17.940369 ip-10-0-129-40 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 01 19:21:17.940383 ip-10-0-129-40 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 01 19:21:17.940393 ip-10-0-129-40 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 01 19:21:17.940732 ip-10-0-129-40 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 01 19:21:27.974002 ip-10-0-129-40 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 01 19:21:27.974051 ip-10-0-129-40 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d399dae81d0c406fa4f925b225510304 --
Jul 01 19:23:22.034463 ip-10-0-129-40 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 19:23:22.458347 ip-10-0-129-40 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 19:23:22.458347 ip-10-0-129-40 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 19:23:22.458347 ip-10-0-129-40 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 19:23:22.458347 ip-10-0-129-40 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 19:23:22.458347 ip-10-0-129-40 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 19:23:22.459749 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.459657    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 19:23:22.465489 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465467    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:23:22.465489 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465486    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:23:22.465489 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465490    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:23:22.465489 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465495    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465500    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465505    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465509    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465512    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465516    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465520    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465525    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465528    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465532    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465535    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465539    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465543    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465546    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465551    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465555    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465558    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465562    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465566    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465570    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:23:22.465738 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465574    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465577    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465582    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465585    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465589    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465593    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465597    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465603    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465609    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465613    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465618    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465623    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465629    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465638    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465644    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465649    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465653    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465659    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465663    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:23:22.466560 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465667    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465671    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465676    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465680    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465685    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465689    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465693    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465696    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465700    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465704    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465709    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465713    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465717    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465721    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465726    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465730    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465734    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465738    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465743    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465748    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:23:22.467388 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465752    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465756    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465760    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465764    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465768    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465773    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465777    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465782    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465786    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465794    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465798    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465802    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465806    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465811    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465815    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465819    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465823    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465829    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465835    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:23:22.468180 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465839    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465844    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465849    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465853    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465857    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465861    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465865    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465869    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465873    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465888    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465893    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465897    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465909    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465913    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465917    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465921    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465925    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.465929    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467160    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467172    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:23:22.468689 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467178    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467182    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467188    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467192    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467196    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467200    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467205    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467209    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467213    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467217    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467221    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467225    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467229    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467233    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467237    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467241    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467246    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467250    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467254    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467258    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:23:22.469187 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467262    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467266    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467273    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467279    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467283    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467289    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467293    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467298    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467302    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467306    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467311    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467315    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467319    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467323    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467327    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467332    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467336    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467341    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467345    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:23:22.469717 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467349    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467353    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467357    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467361    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467365    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467369    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467374    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467378    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467383    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467387    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467391    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467395    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467399    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467402    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467406    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467411    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467415    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467419    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467424    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:23:22.470205 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467428    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467437    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467443    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467447    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467451    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467455    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467459    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467463    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467468    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467472    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467476    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467479    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467483    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467487    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467492    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467495    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467499    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467504    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467507    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467511    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:23:22.470833 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467515    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467519    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467525    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467529    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467533    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467539    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467545    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467550    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467555    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467560    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467564    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467568    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467572    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467578    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467582    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467586    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467591    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467595    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.467599    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468259    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:23:22.471567 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468271    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468274    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468277    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468281    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468284    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468287    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468289    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468292    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468294    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468297    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468300    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468302    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468305    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468307    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468310    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468312    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468315    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468318    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468320    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468323    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:23:22.472097 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468326    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468328    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468331    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468333    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468336    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468338    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468341    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468343    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468354    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468359    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468362    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468365    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468369    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468373    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468375    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468378    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468381    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468384    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468387    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468390    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:23:22.472641 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468392    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468395    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468397    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468400    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468403    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468405    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468410    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468412    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468415    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468417    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468420    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468422    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468425    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468427    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468430    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468432    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468436    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468440    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468443    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:23:22.473145 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468447    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468449    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468452    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468455    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468457    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468461    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468466    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468469    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468472    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468475    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468477    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468481    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468484    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468488    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468491    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468494    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468496    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468498    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468501    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:23:22.473605 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468503    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468506    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468508    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468511    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468513    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468516    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468518    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468520    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468523    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468525    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468528    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468530    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468532    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468535    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468537    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468539    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468542    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468544    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468552    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.468554    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:23:22.474103 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468642    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468654    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468663    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468668    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468678    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468681    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468686    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468691    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468694    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468697    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468701    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468704    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468707    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468710    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468713    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468716    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468719    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468722    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468726    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468728    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468731    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468734    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468737    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468741    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 19:23:22.474602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468744    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468746    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468749    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468752    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468755    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468758    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468764    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468766    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468771    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468774    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468777    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468780    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468783    2566 flags.go:64] FLAG: --enable-server="true"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468786    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468791    2566 flags.go:64] FLAG: --event-burst="100"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468793    2566 flags.go:64] FLAG: --event-qps="50"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468796    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468799    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468802    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468806    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468808    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468811    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468814    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468817    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468820    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 19:23:22.475249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468823    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468825    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468828    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468831    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468834    2566 flags.go:64] FLAG: --feature-gates=""
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468837    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468840    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468843    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468846    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468849    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468852    2566 flags.go:64] FLAG: --help="false"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468855    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468858    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468864    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468867    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468870    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468873    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468877    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468880    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468883    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468886    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468888    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468891    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468894    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 01 19:23:22.475860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468897    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468900    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468902    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468905    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468908    2566 flags.go:64] FLAG: --lock-file=""
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468910    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468913    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468916    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468922    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468925    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468927    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468930    2566 flags.go:64] FLAG: --logging-format="text"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468933    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468936    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468939    2566 flags.go:64] FLAG: --manifest-url=""
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468942    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468946    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468949    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468953    2566 flags.go:64] FLAG: --max-pods="110"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468956    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468959    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468964    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468967    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468970    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468973    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 19:23:22.476471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468976    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468985    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468987    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468990    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468993    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.468996    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469001    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469004    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469007    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469010    2566 flags.go:64] FLAG: --port="10250"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469012    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469015    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03f435c8a1ff46b91"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469018    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469021    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469024    2566 flags.go:64] FLAG: --register-node="true"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469027    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469030    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469033    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469036    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469039    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469042    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469045    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469048    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469050    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469053    2566 flags.go:64] FLAG: --runonce="false"
Jul 01 19:23:22.477080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469056    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469059    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469061    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469066    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469071    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469074    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469077    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469080    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469082    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469088    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469091    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469094    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469097    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469100    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469103    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469108    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469111    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469113    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469134    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469137    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469139    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469142    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469145    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469148    2566 flags.go:64] FLAG: --v="2"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469152    2566 flags.go:64] FLAG: --version="false"
Jul 01 19:23:22.477732 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469156    2566 flags.go:64] FLAG: --vmodule=""
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469159    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469162    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469278    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469283    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469287    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469290    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469293    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469296    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469299    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469303    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469307    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469310    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469313    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469316    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469318    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469321    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469324    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469327    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469329    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:23:22.478357 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469331    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469334    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469337    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469339    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469341    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469344    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469347    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469349    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469352    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469354    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469357    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469360    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469363    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469366    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469369    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469372    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469374    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469377    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469379    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:23:22.478841 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469382    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469384    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469387    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469390    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469394    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469396    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469399    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469401    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469403    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469406    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469408    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469411    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469414    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469416    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469419    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469422    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469426    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469429    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469432    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:23:22.479351 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469434    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469437    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469439    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469441    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469444    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469446    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469448    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469451    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469453    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469456    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469458    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469461    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469463    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469466    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469468    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469470    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469474    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469478    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469481    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469483    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:23:22.479793 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469486    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469488    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469491    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469493    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469496    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469499    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469502    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469510    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469514    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469516    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469519    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469521    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469523    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469526    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469528    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469531    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469533    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469536    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469538    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469541    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:23:22.480311 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469544    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469546    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469549    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.469551    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.469557    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.476908    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.476926    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.476947    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.476953    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477010    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477015    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477018    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477021    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477024    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477027    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:23:22.480807 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477030    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477032    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477035    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477037    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477040    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477043    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477045    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477048    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477050    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477053    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477055    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477058    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477060    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477063    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477065    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477068    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477070    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477073    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477075    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477077    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:23:22.481227 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477081    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477085    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477087    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477091    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477093    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477096    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477099    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477102    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477104    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477106    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477109    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477112    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477131    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477136    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477140    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477143    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477145    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477148    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477150    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:23:22.481708 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477153    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477156    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477158    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477161    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477164    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477166    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477169    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477171    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477174    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477177    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477179    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477182    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477184    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477187    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477189    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477192    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477196    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477199    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477202    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477205    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:23:22.482188 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477209    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477212    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477215    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477219    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477222    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477225    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477227    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477230    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477233    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477236    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477238    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477241    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477244    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477247    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477250    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477253    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477255    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477258    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477261    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477263    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:23:22.482677 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477266    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477269    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477271    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477274    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477276    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477279    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477281    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477283    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477286    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477289    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477291    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477294    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477296    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477299    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.477303    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 19:23:22.483167 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477448    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477453    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477455    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477458    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477461    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477464    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477466    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477469    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477471    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477474    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477476    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477479    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477482    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477484    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477487    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477490    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477493    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477495    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477498    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477500    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:23:22.483533 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477503    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477506    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477508    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477510    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477513    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477515    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477518    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477521    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477523    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477526    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477528    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477531    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477535    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477538    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477541    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477544    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477547    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477549    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477552    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:23:22.484000 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477554    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477557    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477559    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477561    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477564    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477566    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477569    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477572    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477574    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477577    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477579    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477582    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477584    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477587    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477589    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477591    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477593    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477596    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477598    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477601    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:23:22.484467 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477603    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477606    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477608    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477610    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477613    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477615    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477618    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477620    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477622    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477625    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477627    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477629    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477632    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477634    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477636    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477640    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477643    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477646    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477649    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:23:22.484937 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477654    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477657    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477659    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477662    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477664    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477667    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477669    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477672    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477674    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477677    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477679    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477682    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477684    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477686    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477689    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477691    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477693    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477696    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477698    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477701    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:23:22.485403 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:22.477703    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:23:22.485871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.477707    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 19:23:22.485871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.477822    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 19:23:22.485871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.482159    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 19:23:22.485871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.483556    2566 server.go:1019] "Starting client certificate rotation"
Jul 01 19:23:22.485871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.483656    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 19:23:22.485871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.483699    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 19:23:22.486056 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.486043    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 19:23:22.487513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.487500    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 19:23:22.494357 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.494341    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 01 19:23:22.497253 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.497237    2566 log.go:25] "Validated CRI v1 image API"
Jul 01 19:23:22.498110 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.498092    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 19:23:22.502255 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.502232    2566 fs.go:135] Filesystem UUIDs: map[11bcbc8e-80a1-4325-8332-f09f45e2503b:/dev/nvme0n1p3 23316783-e94f-490d-8d45-a271f4351270:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 01 19:23:22.502338 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.502252    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 19:23:22.507738 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.507629    2566 manager.go:216] Machine: {Timestamp:2026-07-01 19:23:22.505982735 +0000 UTC m=+0.347077003 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095128 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2571c8cdc11f8e672c98685fd3ac0f SystemUUID:ec2571c8-cdc1-1f8e-672c-98685fd3ac0f BootID:d399dae8-1d0c-406f-a4f9-25b225510304 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c3:36:9c:39:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c3:36:9c:39:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:22:83:59:e1:4a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 19:23:22.507738 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.507734    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 19:23:22.507846 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.507820    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 19:23:22.511349 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.511320    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 19:23:22.511521 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.511353    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 19:23:22.512192 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.512182    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 19:23:22.512228 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.512196    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 19:23:22.512228 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.512203    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 19:23:22.512714 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.512703    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 19:23:22.513665 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.513654    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 19:23:22.515658 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.515637    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 19:23:22.517151 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517137    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 19:23:22.517278 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517268    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 19:23:22.517319 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517303    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 19:23:22.517319 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517314    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 19:23:22.517377 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517326    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 19:23:22.517377 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517336    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 01 19:23:22.517377 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517345    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 19:23:22.517752 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517742    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 19:23:22.517807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.517764    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 19:23:22.520375 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.520355    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 19:23:22.521869 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.521854    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 19:23:22.521919 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.521886    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 19:23:22.522477 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.522467    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 19:23:22.522513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.522483    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 19:23:22.522513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.522491    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 19:23:22.522513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.522497    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 19:23:22.522513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.522506    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 19:23:22.522513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.522513    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 19:23:22.523080 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523072    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 19:23:22.523109 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523082    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 19:23:22.523109 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523090    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 19:23:22.523109 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523099    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 19:23:22.523109 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523108    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 19:23:22.523226 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523134    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 19:23:22.523918 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523908    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 19:23:22.523952 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.523919    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 19:23:22.527350 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.527336    2566 server.go:1268] "Started kubelet"
Jul 01 19:23:22.527571 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.527539    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 19:23:22.527926 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.527900    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 19:23:22.528009 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.527961    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 19:23:22.528291 ip-10-0-129-40 systemd[1]: Started Kubernetes Kubelet.
Jul 01 19:23:22.529496 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.529192    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 19:23:22.529557 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.529510    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 01 19:23:22.529557 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.529512    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 01 19:23:22.531315 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.530042    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 19:23:22.534133 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.534083    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 01 19:23:22.539428 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.539365    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 19:23:22.539515 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.538493    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-40.ec2.internal.18be40d8d5ad2da5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-40.ec2.internal,UID:ip-10-0-129-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-40.ec2.internal,},FirstTimestamp:2026-07-01 19:23:22.527305125 +0000 UTC m=+0.368399393,LastTimestamp:2026-07-01 19:23:22.527305125 +0000 UTC m=+0.368399393,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-40.ec2.internal,}"
Jul 01 19:23:22.541406 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.541386    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tx29m"
Jul 01 19:23:22.542314 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.542299    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 19:23:22.542376 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.542318    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 19:23:22.542422 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.542412    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:22.542961 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.542944    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 19:23:22.542961 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.542947    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 19:23:22.543099 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.542972    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 19:23:22.543099 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543049    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 19:23:22.543099 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543057    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 19:23:22.543608 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543502    2566 factory.go:55] Registering systemd factory
Jul 01 19:23:22.543608 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543518    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 01 19:23:22.543886 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543797    2566 factory.go:153] Registering CRI-O factory
Jul 01 19:23:22.543886 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543816    2566 factory.go:223] Registration of the crio container factory successfully
Jul 01 19:23:22.543886 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543868    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 19:23:22.544025 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543896    2566 factory.go:103] Registering Raw factory
Jul 01 19:23:22.544025 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.543912    2566 manager.go:1196] Started watching for new ooms in manager
Jul 01 19:23:22.544670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.544642    2566 manager.go:349] Starting recovery of all containers
Jul 01 19:23:22.547595 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.547572    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tx29m"
Jul 01 19:23:22.548950 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.548920    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 01 19:23:22.549045 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.548980    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 01 19:23:22.555165 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.555001    2566 manager.go:354] Recovery completed
Jul 01 19:23:22.559785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.559770    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.562398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.562381    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.562467 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.562414    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.562467 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.562429    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.563445 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.563423    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 19:23:22.563445 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.563436    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 19:23:22.563522 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.563450    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 19:23:22.563522 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.563465    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.564770 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.564758    2566 policy_none.go:49] "None policy: Start"
Jul 01 19:23:22.564809 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.564775    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 19:23:22.564809 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.564785    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 19:23:22.565830 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.565819    2566 policy_none.go:47] "Start"
Jul 01 19:23:22.568084 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.568070    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.568507 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.568344    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.568507 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.568393    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.611765 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.611744    2566 manager.go:337] "Starting Device Plugin manager"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.611783    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.611796    2566 server.go:87] "Starting device plugin registration server"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.612057    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.612184    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.612295    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.612407    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.612416    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.612977    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.613021    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.618814    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.620094    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.620148    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.620176    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 19:23:22.620335 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.620218    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 19:23:22.622925 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.622908    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:23:22.712383 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.712280    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.713353 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.713335    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.713464 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.713374    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.713464 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.713389    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.713464 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.713419    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.720497 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.720472    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"]
Jul 01 19:23:22.720584 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.720544    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.722077 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.722060    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.722191 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.722092    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.722191 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.722104    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.723624 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.723609    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.723758 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.723721    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.723758 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.723747    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.724405 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.724383    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.724503 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.724415    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.724503 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.724429    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.724503 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.724485    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.724649 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.724512    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.724649 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.724525    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.725590 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.725577    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.725643 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.725596    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-40.ec2.internal\": node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:22.725840 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.725824    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.725916 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.725854    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:23:22.726598 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.726572    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:23:22.726702 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.726601    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:23:22.726702 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.726614    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:23:22.744945 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.744922    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/851c30434749251be19b556dd27c3891-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"851c30434749251be19b556dd27c3891\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.745050 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.744968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6ef9212a9de236e367e5cfa4ea34ab4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"f6ef9212a9de236e367e5cfa4ea34ab4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.745050 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.745006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6ef9212a9de236e367e5cfa4ea34ab4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"f6ef9212a9de236e367e5cfa4ea34ab4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.747980 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.747954    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:22.750952 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.750935    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-40.ec2.internal\" not found" node="ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.755426 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.755410    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-40.ec2.internal\" not found" node="ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.845471 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.845434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6ef9212a9de236e367e5cfa4ea34ab4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"f6ef9212a9de236e367e5cfa4ea34ab4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.845653 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.845484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6ef9212a9de236e367e5cfa4ea34ab4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"f6ef9212a9de236e367e5cfa4ea34ab4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.845653 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.845535    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6ef9212a9de236e367e5cfa4ea34ab4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"f6ef9212a9de236e367e5cfa4ea34ab4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.845653 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.845539    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6ef9212a9de236e367e5cfa4ea34ab4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"f6ef9212a9de236e367e5cfa4ea34ab4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.845653 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.845568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/851c30434749251be19b556dd27c3891-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"851c30434749251be19b556dd27c3891\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.845653 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:22.845608    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/851c30434749251be19b556dd27c3891-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"851c30434749251be19b556dd27c3891\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:22.848562 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.848544    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:22.949000 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:22.948974    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:23.049350 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:23.049296    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:23.054462 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:23.054445    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:23.058840 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:23.058824    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:23.149518 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:23.149488    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:24.542382 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:24.542337    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:24.642883 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:24.642798    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:24.743383 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:24.743339    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:24.775505 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.775477    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 19:23:24.776027 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.775634    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 19:23:24.844031 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:24.844006    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found"
Jul 01 19:23:24.844031 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.844023    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 19:18:23 +0000 UTC" deadline="2028-02-24 00:51:20.923665866 +0000 UTC"
Jul 01 19:23:24.844226 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.844049    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14453h27m56.079620444s"
Jul 01 19:23:24.889358 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:24.889309    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ef9212a9de236e367e5cfa4ea34ab4.slice/crio-20d03f92439095d47934d440febe98c55548f8d483b2f196911597b3509807c5 WatchSource:0}: Error finding container 20d03f92439095d47934d440febe98c55548f8d483b2f196911597b3509807c5: Status 404 returned error can't find the container with id 20d03f92439095d47934d440febe98c55548f8d483b2f196911597b3509807c5
Jul 01 19:23:24.889624 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:24.889604    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851c30434749251be19b556dd27c3891.slice/crio-98bbcf74464b42241e06ad701a627e9be863d97bacbf029ca01ece47e83fba5b WatchSource:0}: Error finding container 98bbcf74464b42241e06ad701a627e9be863d97bacbf029ca01ece47e83fba5b: Status 404 returned error can't find the container with id 98bbcf74464b42241e06ad701a627e9be863d97bacbf029ca01ece47e83fba5b
Jul 01 19:23:24.896234 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.896211    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:23:24.916314 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.916279    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"f6ef9212a9de236e367e5cfa4ea34ab4","Type":"ContainerStarted","Data":"20d03f92439095d47934d440febe98c55548f8d483b2f196911597b3509807c5"}
Jul 01 19:23:24.917382 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.917344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" event={"ID":"851c30434749251be19b556dd27c3891","Type":"ContainerStarted","Data":"98bbcf74464b42241e06ad701a627e9be863d97bacbf029ca01ece47e83fba5b"}
Jul 01 19:23:24.933711 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.933688    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:23:24.935722 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.935707    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:24.948277 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.948256    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 19:23:24.950207 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.950191    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"
Jul 01 19:23:24.963120 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:24.963106    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 19:23:25.090252 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.090216    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:23:25.424918 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.424887    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:23:25.810591 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.810510    2566 apiserver.go:52] "Watching apiserver"
Jul 01 19:23:25.817403 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.817356    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 19:23:25.818879 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.818849    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4wtck","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq","openshift-cluster-node-tuning-operator/tuned-rxhp8","openshift-multus/network-metrics-daemon-qw5hn","openshift-network-operator/iptables-alerter-hnm9j","openshift-ovn-kubernetes/ovnkube-node-gcgrj","kube-system/konnectivity-agent-zzn2q","kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal","openshift-dns/node-resolver-dkmcn","openshift-image-registry/node-ca-zxrk2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal","openshift-multus/multus-9njfs","openshift-multus/multus-additional-cni-plugins-zmrdr","openshift-network-diagnostics/network-check-target-bdsqr"]
Jul 01 19:23:25.820666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.820641    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.820770 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.820729    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:25.822237 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.822205    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.823798 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.823780    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.824557 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.824528    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 19:23:25.824668 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.824585    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.824728 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.824682    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p7grv\""
Jul 01 19:23:25.824728 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.824591    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.825658 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.825637    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:25.825757 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.825731    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:25.825851 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.825777    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.825921 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.825903    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-krxcm\""
Jul 01 19:23:25.825979 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.825969    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.827408 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.827391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.829264 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.829245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.829602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.829425    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 19:23:25.829602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.829557    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.829748 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.829725    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nnkng\""
Jul 01 19:23:25.829819 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.829742    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.831142 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.831124    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.831766 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.831748    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dxmdb\""
Jul 01 19:23:25.831766 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.831763    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 19:23:25.831930 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.831763    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.831930 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.831751    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 19:23:25.832709 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.832689    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 19:23:25.832801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.832689    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.832801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.832697    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 19:23:25.833060 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.833043    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.833335 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.833315    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 19:23:25.833647 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.833627    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 19:23:25.833947 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.833922    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z8tsm\""
Jul 01 19:23:25.834949 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.834929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.835409 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.835391    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 19:23:25.835509 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.835476    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.835799 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.835778    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.835897 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.835789    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w2jtg\""
Jul 01 19:23:25.837235 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.837217    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.837554 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.837540    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.838125 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.837787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 19:23:25.838125 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.837928    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.838125 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.837965    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zkvcp\""
Jul 01 19:23:25.839382 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.839346    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 19:23:25.839806 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.839787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 19:23:25.840503 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.840485    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 19:23:25.840612 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.840493    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 19:23:25.840673 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.840640    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-flb8r\""
Jul 01 19:23:25.841195 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.841179    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.843800 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.843556    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:25.843800 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.843621    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:25.844105 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.844073    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 19:23:25.844245 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.844232    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 19:23:25.844296 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.844245    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cxvcs\""
Jul 01 19:23:25.845502 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-env-overrides\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.845502 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845429    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-kubelet\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.845502 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845487    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-hostroot\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.845670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845515    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.845670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845539    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-modprobe-d\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.845670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845562    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-ovn\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.845670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845586    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v28vg\" (UniqueName: \"kubernetes.io/projected/193f723e-55a0-49d9-9f96-d222090fb79d-kube-api-access-v28vg\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.845670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845608    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-device-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.845670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845655    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-sys-fs\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-systemd-units\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-cni-netd\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845755    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q6pg\" (UniqueName: \"kubernetes.io/projected/8dc0ae12-7f88-49fe-8193-2c103f614edb-kube-api-access-6q6pg\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-kubelet-config\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845785    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-cni-binary-copy\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-kubernetes\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsn4k\" (UniqueName: \"kubernetes.io/projected/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-kube-api-access-gsn4k\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:25.845860 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845854    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-etc-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845893    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ac5527dc-647a-4265-a9a9-d393f325753d-agent-certs\") pod \"konnectivity-agent-zzn2q\" (UID: \"ac5527dc-647a-4265-a9a9-d393f325753d\") " pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/246949f7-3788-4e6f-8561-67b3ed764b7d-iptables-alerter-script\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovnkube-script-lib\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.845991    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-netns\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-multus-certs\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frndt\" (UniqueName: \"kubernetes.io/projected/364edccc-1262-4d66-9533-ae4bd8a2c8b0-kube-api-access-frndt\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-daemon-config\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-etc-kubernetes\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846138    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-systemd\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846173    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmbz8\" (UniqueName: \"kubernetes.io/projected/62099bca-1e18-41da-b8c0-968bde285901-kube-api-access-hmbz8\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846208    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-node-log\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovn-node-metrics-cert\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e22d576-1039-4788-a13d-a98075ea84b4-tmp-dir\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-cni-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846334    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-socket-dir-parent\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysconfig\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-sys\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-var-lib-kubelet\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-log-socket\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846471    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-cni-multus\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846493    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-conf-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846517    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/193f723e-55a0-49d9-9f96-d222090fb79d-host\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846567    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-os-release\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62099bca-1e18-41da-b8c0-968bde285901-etc-tuned\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62099bca-1e18-41da-b8c0-968bde285901-tmp\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.846720 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846632    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/246949f7-3788-4e6f-8561-67b3ed764b7d-host-slash\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846648    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e22d576-1039-4788-a13d-a98075ea84b4-hosts-file\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846678    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjp4l\" (UniqueName: \"kubernetes.io/projected/0e22d576-1039-4788-a13d-a98075ea84b4-kube-api-access-tjp4l\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/193f723e-55a0-49d9-9f96-d222090fb79d-serviceca\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-cnibin\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846761    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glfff\" (UniqueName: \"kubernetes.io/projected/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-kube-api-access-glfff\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846785    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-kubelet\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-slash\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-var-lib-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-k8s-cni-cncf-io\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-cni-bin\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-socket-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846912    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-host\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846938    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-run-netns\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846961    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovnkube-config\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.846996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.847328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847025    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwz4c\" (UniqueName: \"kubernetes.io/projected/246949f7-3788-4e6f-8561-67b3ed764b7d-kube-api-access-gwz4c\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-systemd\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-registration-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847121    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysctl-d\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-lib-modules\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847162    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847184    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ac5527dc-647a-4265-a9a9-d393f325753d-konnectivity-ca\") pod \"konnectivity-agent-zzn2q\" (UID: \"ac5527dc-647a-4265-a9a9-d393f325753d\") " pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysctl-conf\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-cni-bin\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847252    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-system-cni-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-run\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.847923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.847294    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-dbus\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.872599 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.872565    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k9mf9"
Jul 01 19:23:25.878863 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.878843    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k9mf9"
Jul 01 19:23:25.936298 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.936271    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 19:23:25.948164 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-cni-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-socket-dir-parent\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysconfig\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-sys\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-var-lib-kubelet\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-log-socket\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-cni-multus\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948291    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-socket-dir-parent\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948350    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-var-lib-kubelet\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948352 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948353    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-log-socket\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948347    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-cni-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948387    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-sys\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948409    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysconfig\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948420    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-cni-multus\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-conf-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948482    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948509    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/193f723e-55a0-49d9-9f96-d222090fb79d-host\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948524    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948485    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-conf-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-os-release\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948569    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948585    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-os-release\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft8z9\" (UniqueName: \"kubernetes.io/projected/2a2b3861-694a-4821-bda3-2d6f27e7567a-kube-api-access-ft8z9\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948625    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/193f723e-55a0-49d9-9f96-d222090fb79d-host\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62099bca-1e18-41da-b8c0-968bde285901-etc-tuned\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62099bca-1e18-41da-b8c0-968bde285901-tmp\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.948785 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948666    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/246949f7-3788-4e6f-8561-67b3ed764b7d-host-slash\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e22d576-1039-4788-a13d-a98075ea84b4-hosts-file\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjp4l\" (UniqueName: \"kubernetes.io/projected/0e22d576-1039-4788-a13d-a98075ea84b4-kube-api-access-tjp4l\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948739    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/246949f7-3788-4e6f-8561-67b3ed764b7d-host-slash\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/193f723e-55a0-49d9-9f96-d222090fb79d-serviceca\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948770    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-cnibin\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948792    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glfff\" (UniqueName: \"kubernetes.io/projected/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-kube-api-access-glfff\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948812    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e22d576-1039-4788-a13d-a98075ea84b4-hosts-file\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-kubelet\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-slash\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-var-lib-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948897    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-cnibin\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-k8s-cni-cncf-io\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-cni-bin\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948956    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-kubelet\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948961    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-socket-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.948995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-host\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949021    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-k8s-cni-cncf-io\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.949601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-run-netns\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949084    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-run-netns\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949086    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-var-lib-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949093    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-socket-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949088    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-host\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949080    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949120    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-slash\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949129    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-cni-bin\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949131    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovnkube-config\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949173    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwz4c\" (UniqueName: \"kubernetes.io/projected/246949f7-3788-4e6f-8561-67b3ed764b7d-kube-api-access-gwz4c\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-systemd\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949236    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-registration-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949251    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysctl-d\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949266    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-lib-modules\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.950589 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ac5527dc-647a-4265-a9a9-d393f325753d-konnectivity-ca\") pod \"konnectivity-agent-zzn2q\" (UID: \"ac5527dc-647a-4265-a9a9-d393f325753d\") " pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysctl-conf\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949317    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-cni-bin\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-system-cni-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949388    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-systemd\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-run\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-dbus\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.949438    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-env-overrides\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.949489    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:23:26.449474878 +0000 UTC m=+2.998321231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-kubelet\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-hostroot\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949560    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949575    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysctl-d\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.951361 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949441    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-run\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949638    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-var-lib-kubelet\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949643    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovnkube-config\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949669    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-hostroot\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949679    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-lib-modules\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949278    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/193f723e-55a0-49d9-9f96-d222090fb79d-serviceca\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949803    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-env-overrides\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949818    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-sysctl-conf\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949859    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-registration-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949889    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949916    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-system-cni-dir\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949920    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-dbus\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949577    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-cnibin\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949961    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-cni-bin\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.949997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-modprobe-d\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-ovn\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950056    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-run-ovn\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950105    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-modprobe-d\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950134    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950144    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v28vg\" (UniqueName: \"kubernetes.io/projected/193f723e-55a0-49d9-9f96-d222090fb79d-kube-api-access-v28vg\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950178    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-device-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-sys-fs\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950259    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ac5527dc-647a-4265-a9a9-d393f325753d-konnectivity-ca\") pod \"konnectivity-agent-zzn2q\" (UID: \"ac5527dc-647a-4265-a9a9-d393f325753d\") " pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950270    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-device-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950307    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950332    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-sys-fs\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-systemd-units\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950416    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-systemd-units\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950459    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-cni-netd\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.950489    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950532    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-host-cni-netd\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:25.950595    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:23:26.450579333 +0000 UTC m=+2.999425686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q6pg\" (UniqueName: \"kubernetes.io/projected/8dc0ae12-7f88-49fe-8193-2c103f614edb-kube-api-access-6q6pg\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.950983    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-kubelet-config\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.952955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-cni-binary-copy\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-kubernetes\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951056    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-kubelet-config\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951063    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn4k\" (UniqueName: \"kubernetes.io/projected/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-kube-api-access-gsn4k\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-etc-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951123    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-kubernetes\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951161    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-etc-openvswitch\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951125    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ac5527dc-647a-4265-a9a9-d393f325753d-agent-certs\") pod \"konnectivity-agent-zzn2q\" (UID: \"ac5527dc-647a-4265-a9a9-d393f325753d\") " pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951200    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951227    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/246949f7-3788-4e6f-8561-67b3ed764b7d-iptables-alerter-script\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovnkube-script-lib\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951282    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-netns\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-multus-certs\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951339    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-os-release\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frndt\" (UniqueName: \"kubernetes.io/projected/364edccc-1262-4d66-9533-ae4bd8a2c8b0-kube-api-access-frndt\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951438    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-daemon-config\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.953678 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-etc-kubernetes\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951503    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-netns\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951507    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-systemd\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951541    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmbz8\" (UniqueName: \"kubernetes.io/projected/62099bca-1e18-41da-b8c0-968bde285901-kube-api-access-hmbz8\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951552    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62099bca-1e18-41da-b8c0-968bde285901-etc-systemd\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-node-log\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951594    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovn-node-metrics-cert\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951610    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/364edccc-1262-4d66-9533-ae4bd8a2c8b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e22d576-1039-4788-a13d-a98075ea84b4-tmp-dir\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951756    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-etc-kubernetes\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951758    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovnkube-script-lib\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951820    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-host-run-multus-certs\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951860    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dc0ae12-7f88-49fe-8193-2c103f614edb-node-log\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.951904    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e22d576-1039-4788-a13d-a98075ea84b4-tmp-dir\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.952413    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-cni-binary-copy\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.952437    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62099bca-1e18-41da-b8c0-968bde285901-etc-tuned\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.952452    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-multus-daemon-config\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.954034    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62099bca-1e18-41da-b8c0-968bde285901-tmp\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.954448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.954131    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/246949f7-3788-4e6f-8561-67b3ed764b7d-iptables-alerter-script\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.955181 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.955067    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dc0ae12-7f88-49fe-8193-2c103f614edb-ovn-node-metrics-cert\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:25.955233 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.955203    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ac5527dc-647a-4265-a9a9-d393f325753d-agent-certs\") pod \"konnectivity-agent-zzn2q\" (UID: \"ac5527dc-647a-4265-a9a9-d393f325753d\") " pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:25.961465 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.961014    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwz4c\" (UniqueName: \"kubernetes.io/projected/246949f7-3788-4e6f-8561-67b3ed764b7d-kube-api-access-gwz4c\") pod \"iptables-alerter-hnm9j\" (UID: \"246949f7-3788-4e6f-8561-67b3ed764b7d\") " pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:25.961465 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.961218    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glfff\" (UniqueName: \"kubernetes.io/projected/7f93fdf9-24f9-44ff-8e51-583b8d5b8c30-kube-api-access-glfff\") pod \"multus-9njfs\" (UID: \"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30\") " pod="openshift-multus/multus-9njfs"
Jul 01 19:23:25.961815 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.961635    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmbz8\" (UniqueName: \"kubernetes.io/projected/62099bca-1e18-41da-b8c0-968bde285901-kube-api-access-hmbz8\") pod \"tuned-rxhp8\" (UID: \"62099bca-1e18-41da-b8c0-968bde285901\") " pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:25.962441 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.962411    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frndt\" (UniqueName: \"kubernetes.io/projected/364edccc-1262-4d66-9533-ae4bd8a2c8b0-kube-api-access-frndt\") pod \"aws-ebs-csi-driver-node-zgvbq\" (UID: \"364edccc-1262-4d66-9533-ae4bd8a2c8b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:25.963788 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.963766    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsn4k\" (UniqueName: \"kubernetes.io/projected/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-kube-api-access-gsn4k\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:25.963788 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.963784    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjp4l\" (UniqueName: \"kubernetes.io/projected/0e22d576-1039-4788-a13d-a98075ea84b4-kube-api-access-tjp4l\") pod \"node-resolver-dkmcn\" (UID: \"0e22d576-1039-4788-a13d-a98075ea84b4\") " pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:25.964719 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.964697    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v28vg\" (UniqueName: \"kubernetes.io/projected/193f723e-55a0-49d9-9f96-d222090fb79d-kube-api-access-v28vg\") pod \"node-ca-zxrk2\" (UID: \"193f723e-55a0-49d9-9f96-d222090fb79d\") " pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:25.965213 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:25.965158    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q6pg\" (UniqueName: \"kubernetes.io/projected/8dc0ae12-7f88-49fe-8193-2c103f614edb-kube-api-access-6q6pg\") pod \"ovnkube-node-gcgrj\" (UID: \"8dc0ae12-7f88-49fe-8193-2c103f614edb\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:26.052096 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052056    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052275 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052321 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052299    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052351 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052326    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052351 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-cnibin\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052431 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:26.052431 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052415    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-os-release\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052441    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052461    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052476    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft8z9\" (UniqueName: \"kubernetes.io/projected/2a2b3861-694a-4821-bda3-2d6f27e7567a-kube-api-access-ft8z9\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052644 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052624    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052731 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052718    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-os-release\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052828 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052814    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.052864 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.052848    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a2b3861-694a-4821-bda3-2d6f27e7567a-cnibin\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.053054 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.053041    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.053156 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.053141    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a2b3861-694a-4821-bda3-2d6f27e7567a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.066005 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.065934    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:26.066005 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.065964    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:26.066005 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.065978    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:26.066220 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.066045    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:26.566027131 +0000 UTC m=+3.114873490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:26.067166 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.067144    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft8z9\" (UniqueName: \"kubernetes.io/projected/2a2b3861-694a-4821-bda3-2d6f27e7567a-kube-api-access-ft8z9\") pod \"multus-additional-cni-plugins-zmrdr\" (UID: \"2a2b3861-694a-4821-bda3-2d6f27e7567a\") " pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.134539 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.134502    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq"
Jul 01 19:23:26.141542 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.141518    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rxhp8"
Jul 01 19:23:26.150433 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.150416    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hnm9j"
Jul 01 19:23:26.156134 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.156113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:26.161616 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.161599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:26.167906 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.167890    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dkmcn"
Jul 01 19:23:26.175378 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.175340    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zxrk2"
Jul 01 19:23:26.180938 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.180919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9njfs"
Jul 01 19:23:26.186455 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.186433    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zmrdr"
Jul 01 19:23:26.215484 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.215464    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:23:26.455109 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.455038    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:26.455109 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.455085    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:26.455266 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.455202    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:26.455266 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.455258    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:23:27.45524357 +0000 UTC m=+4.004089923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:26.455337 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.455202    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:26.455337 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.455319    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:23:27.455307384 +0000 UTC m=+4.004153740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:26.658680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.657574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:26.658680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.657751    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:26.658680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.657775    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:26.658680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.657786    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:26.658680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:26.657839    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:27.65782415 +0000 UTC m=+4.206670502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:26.776936 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:26.776911    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc0ae12_7f88_49fe_8193_2c103f614edb.slice/crio-71ef7f6082061261cd033dc527de310deada1e9addd928cf5b478545d1e265c9 WatchSource:0}: Error finding container 71ef7f6082061261cd033dc527de310deada1e9addd928cf5b478545d1e265c9: Status 404 returned error can't find the container with id 71ef7f6082061261cd033dc527de310deada1e9addd928cf5b478545d1e265c9
Jul 01 19:23:26.778219 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:26.778193    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e22d576_1039_4788_a13d_a98075ea84b4.slice/crio-63c5e5c02d71e17b09eb4d7253f867f6b45f64bd6dbe3d779b80eb8bd7d0f4b0 WatchSource:0}: Error finding container 63c5e5c02d71e17b09eb4d7253f867f6b45f64bd6dbe3d779b80eb8bd7d0f4b0: Status 404 returned error can't find the container with id 63c5e5c02d71e17b09eb4d7253f867f6b45f64bd6dbe3d779b80eb8bd7d0f4b0
Jul 01 19:23:26.778965 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:26.778946    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364edccc_1262_4d66_9533_ae4bd8a2c8b0.slice/crio-58a2b2ae199750d1264d889d52b6171576f0c2630345121078546bcd962d6ac2 WatchSource:0}: Error finding container 58a2b2ae199750d1264d889d52b6171576f0c2630345121078546bcd962d6ac2: Status 404 returned error can't find the container with id 58a2b2ae199750d1264d889d52b6171576f0c2630345121078546bcd962d6ac2
Jul 01 19:23:26.799569 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:26.799550    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2b3861_694a_4821_bda3_2d6f27e7567a.slice/crio-a0919e0fe8320da3d299070fd0fc18343b30f298bcbf5fbc7194c7774f9eb9ec WatchSource:0}: Error finding container a0919e0fe8320da3d299070fd0fc18343b30f298bcbf5fbc7194c7774f9eb9ec: Status 404 returned error can't find the container with id a0919e0fe8320da3d299070fd0fc18343b30f298bcbf5fbc7194c7774f9eb9ec
Jul 01 19:23:26.806853 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:26.806757    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62099bca_1e18_41da_b8c0_968bde285901.slice/crio-cc46604b8637cde574321c85ba158557c34197f3d2234f06842691e995c431c2 WatchSource:0}: Error finding container cc46604b8637cde574321c85ba158557c34197f3d2234f06842691e995c431c2: Status 404 returned error can't find the container with id cc46604b8637cde574321c85ba158557c34197f3d2234f06842691e995c431c2
Jul 01 19:23:26.807932 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:23:26.807882    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246949f7_3788_4e6f_8561_67b3ed764b7d.slice/crio-fe12813be4fe009902e5a9035cf5ecef3cbbedb01f8645cf3fbbb80aef2e0048 WatchSource:0}: Error finding container fe12813be4fe009902e5a9035cf5ecef3cbbedb01f8645cf3fbbb80aef2e0048: Status 404 returned error can't find the container with id fe12813be4fe009902e5a9035cf5ecef3cbbedb01f8645cf3fbbb80aef2e0048
Jul 01 19:23:26.882128 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.881947    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:18:25 +0000 UTC" deadline="2028-03-20 11:38:35.038857664 +0000 UTC"
Jul 01 19:23:26.882532 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.882130    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15064h15m8.156732704s"
Jul 01 19:23:26.922864 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.922816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" event={"ID":"851c30434749251be19b556dd27c3891","Type":"ContainerStarted","Data":"113f939e0521c58b3015c0b807b6cc38103e6bdff1fe6c9d8add172cf5e937e7"}
Jul 01 19:23:26.924075 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.924048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rxhp8" event={"ID":"62099bca-1e18-41da-b8c0-968bde285901","Type":"ContainerStarted","Data":"cc46604b8637cde574321c85ba158557c34197f3d2234f06842691e995c431c2"}
Jul 01 19:23:26.925410 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.925387    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"71ef7f6082061261cd033dc527de310deada1e9addd928cf5b478545d1e265c9"}
Jul 01 19:23:26.926666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.926641    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hnm9j" event={"ID":"246949f7-3788-4e6f-8561-67b3ed764b7d","Type":"ContainerStarted","Data":"fe12813be4fe009902e5a9035cf5ecef3cbbedb01f8645cf3fbbb80aef2e0048"}
Jul 01 19:23:26.927786 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.927746    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zxrk2" event={"ID":"193f723e-55a0-49d9-9f96-d222090fb79d","Type":"ContainerStarted","Data":"c4bab3de920acec95afca307f53d42c3905a8af5660c7d3e643624481720ab36"}
Jul 01 19:23:26.928941 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.928898    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zzn2q" event={"ID":"ac5527dc-647a-4265-a9a9-d393f325753d","Type":"ContainerStarted","Data":"000fd3d62d61f6b2052b9726544b5b8d0bd4c497d14f0284388d70ebe77d8b38"}
Jul 01 19:23:26.933475 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.933446    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9njfs" event={"ID":"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30","Type":"ContainerStarted","Data":"932e5ec729dc3dc9d1d97b3ef06e08839d4650c8e87be41a53c8e42cb617b4bf"}
Jul 01 19:23:26.935072 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.935051    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerStarted","Data":"a0919e0fe8320da3d299070fd0fc18343b30f298bcbf5fbc7194c7774f9eb9ec"}
Jul 01 19:23:26.936187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.936166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq" event={"ID":"364edccc-1262-4d66-9533-ae4bd8a2c8b0","Type":"ContainerStarted","Data":"58a2b2ae199750d1264d889d52b6171576f0c2630345121078546bcd962d6ac2"}
Jul 01 19:23:26.937270 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.937248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dkmcn" event={"ID":"0e22d576-1039-4788-a13d-a98075ea84b4","Type":"ContainerStarted","Data":"63c5e5c02d71e17b09eb4d7253f867f6b45f64bd6dbe3d779b80eb8bd7d0f4b0"}
Jul 01 19:23:26.937836 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:26.937792    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" podStartSLOduration=2.9377788259999997 podStartE2EDuration="2.937778826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:26.937237644 +0000 UTC m=+3.486084022" watchObservedRunningTime="2026-07-01 19:23:26.937778826 +0000 UTC m=+3.486625202"
Jul 01 19:23:27.462740 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.462653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:27.462893 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.462748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:27.462963 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.462898    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:27.462963 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.462957    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:23:29.462938658 +0000 UTC m=+6.011785016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:27.463442 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.463423    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:27.463537 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.463478    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:23:29.463462306 +0000 UTC m=+6.012308663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:27.663565 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.663527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:27.663742 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.663695    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:27.663742 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.663721    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:27.663742 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.663736    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:27.663902 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.663796    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:29.663777682 +0000 UTC m=+6.212624047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:27.882664 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.882518    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:18:25 +0000 UTC" deadline="2027-12-10 01:01:25.813897968 +0000 UTC"
Jul 01 19:23:27.882664 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.882579    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12629h37m57.931322393s"
Jul 01 19:23:27.916531 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.915723    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:27.916531 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.915857    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:27.916531 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.916221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:27.916531 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.916314    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:27.916531 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.916393    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:27.916531 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:27.916458    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:27.945458 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.945424    2566 generic.go:356] "Generic (PLEG): container finished" podID="f6ef9212a9de236e367e5cfa4ea34ab4" containerID="d789dfba41090dd312e6d2bb84904a10099994a0adcfbc7068b642eeac3ea617" exitCode=0
Jul 01 19:23:27.946332 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:27.946306    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"f6ef9212a9de236e367e5cfa4ea34ab4","Type":"ContainerDied","Data":"d789dfba41090dd312e6d2bb84904a10099994a0adcfbc7068b642eeac3ea617"}
Jul 01 19:23:28.950884 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:28.950848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"f6ef9212a9de236e367e5cfa4ea34ab4","Type":"ContainerStarted","Data":"a12b1ae545e29feef1e71e57adb376c56c0fe5d0ca8af18315f875c912ec0c36"}
Jul 01 19:23:29.480021 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:29.479988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:29.480218 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:29.480046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:29.480218 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.480188    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:29.480329 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.480244    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:23:33.480226089 +0000 UTC m=+10.029072447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:29.480884 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.480857    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:29.481001 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.480924    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:23:33.480907923 +0000 UTC m=+10.029754280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:29.681434 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:29.681380    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:29.681620 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.681517    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:29.681620 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.681548    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:29.681620 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.681564    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:29.681856 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.681625    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:33.681605443 +0000 UTC m=+10.230451810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:29.915508 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:29.915430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:29.915657 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.915603    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:29.915981 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:29.915962    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:29.916080 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.916060    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:29.916123 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:29.916112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:29.916209 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:29.916191    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:31.913904 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:31.913829    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:31.914328 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:31.913936    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:31.914328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:31.914312    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:31.914456 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:31.914437    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:31.914613 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:31.914597    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:31.914715 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:31.914695    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:33.515399 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:33.515337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:33.515813 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:33.515474    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:33.515813 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.515597    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:33.515813 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.515659    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:23:41.515641342 +0000 UTC m=+18.064487699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:33.516104 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.516080    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:33.516183 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.516133    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:23:41.516117806 +0000 UTC m=+18.064964162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:33.717297 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:33.717249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:33.717540 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.717441    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:33.717540 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.717462    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:33.717540 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.717473    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:33.717540 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.717518    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:41.717505205 +0000 UTC m=+18.266351558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:33.917173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:33.916349    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:33.917173 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.916496    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:33.917173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:33.916890    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:33.917173 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.916988    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:33.917173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:33.917033    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:33.917173 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:33.917098    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:35.913071 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:35.913033    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:35.913527 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:35.913082    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:35.913527 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:35.913041    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:35.913527 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:35.913177    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:35.913527 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:35.913325    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:35.913527 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:35.913406    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:37.913531 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:37.913499    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:37.913990 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:37.913499    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:37.913990 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:37.913499    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:37.913990 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:37.913650    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:37.913990 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:37.913718    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:37.913990 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:37.913791    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:39.913162 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:39.913123    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:39.913162 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:39.913147    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:39.913672 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:39.913123    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:39.913672 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:39.913259    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:39.913672 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:39.913336    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:39.913672 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:39.913439    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:41.572674 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:41.572641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:41.573307 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:41.572706    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:41.573307 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.572803    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:41.573307 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.572803    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:41.573307 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.572868    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:23:57.57285356 +0000 UTC m=+34.121699914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:41.573307 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.572884    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:23:57.572877046 +0000 UTC m=+34.121723400 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:41.774415 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:41.774382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:41.774568 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.774507    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:41.774568 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.774523    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:41.774568 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.774533    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:41.774671 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.774583    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:57.774570081 +0000 UTC m=+34.323416434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:41.912948 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:41.912912    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:41.913106 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:41.913035    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:41.913106 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.913045    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:41.913211 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.913156    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:41.913211 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:41.913206    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:41.913315 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:41.913296    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:43.913734 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:43.913703    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:43.914178 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:43.913802    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:43.914178 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:43.913830    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:43.914178 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:43.913859    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:43.914178 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:43.913944    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:43.914178 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:43.914037    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:45.913575 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:45.913547    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:45.913575 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:45.913563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:45.914043 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:45.913570    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:45.914043 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:45.913676    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:45.914043 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:45.913751    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:45.914043 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:45.913804    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:45.985619 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:45.985402    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerStarted","Data":"d0f4243889a7808117604f022ade9863fada601e0a7128acb6cfec1744e5023f"}
Jul 01 19:23:45.988018 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:45.987985    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dkmcn" event={"ID":"0e22d576-1039-4788-a13d-a98075ea84b4","Type":"ContainerStarted","Data":"91fca5ef9cc96b0c5ac6172bc9892fe2052bd2f9fad0449e8a0aa04aa43a0e96"}
Jul 01 19:23:46.008274 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.008222    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" podStartSLOduration=22.008203544 podStartE2EDuration="22.008203544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:28.969290578 +0000 UTC m=+5.518136955" watchObservedRunningTime="2026-07-01 19:23:46.008203544 +0000 UTC m=+22.557049919"
Jul 01 19:23:46.994647 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.994380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rxhp8" event={"ID":"62099bca-1e18-41da-b8c0-968bde285901","Type":"ContainerStarted","Data":"255844c03d1c345c5dfbcf89b6e7950b094fc9666c94a0c821843b2033dc5ec2"}
Jul 01 19:23:46.997001 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.996981    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:23:46.997268 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997248    2566 generic.go:356] "Generic (PLEG): container finished" podID="8dc0ae12-7f88-49fe-8193-2c103f614edb" containerID="7f7c26b3b6388787eb9a1e1924c2ea9f5a6b9787129865bec3b4942aab5a8497" exitCode=1
Jul 01 19:23:46.997346 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997318    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"5f9851c1302ba91d0ae6a6ba41b6a66a452da96475a6a7f8a17cf9bfdcc295a4"}
Jul 01 19:23:46.997404 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997390    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"c99021cf31d05bfad16d79459a68e230b2bf495a07a3e5ed749d3eab26bfde78"}
Jul 01 19:23:46.997451 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"821f06de8a7640749f66f5e0601a9a7d45cdbec6c66e2c38868b2ff14cbaa3ee"}
Jul 01 19:23:46.997501 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997487    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"70ed578d2446a6919ab0eb75d52233556573e5b0749a966e82c059905a7fdce3"}
Jul 01 19:23:46.997553 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997520    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerDied","Data":"7f7c26b3b6388787eb9a1e1924c2ea9f5a6b9787129865bec3b4942aab5a8497"}
Jul 01 19:23:46.997599 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.997560    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"ce67306b4807b0f57896b097dcc8a8bd1e916ba64a525aff4df5b78468b90056"}
Jul 01 19:23:46.998719 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:46.998694    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zxrk2" event={"ID":"193f723e-55a0-49d9-9f96-d222090fb79d","Type":"ContainerStarted","Data":"25c68b884ff15c4ea3c9b8dd0736c19b2a9a48a7c86a428e1ac043bc9d5ea732"}
Jul 01 19:23:47.000040 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.000022    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zzn2q" event={"ID":"ac5527dc-647a-4265-a9a9-d393f325753d","Type":"ContainerStarted","Data":"b2efcef0ec0f752f42816167b91985c7f6d659d588a8f61089ac9c2428582ea5"}
Jul 01 19:23:47.001457 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.001440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9njfs" event={"ID":"7f93fdf9-24f9-44ff-8e51-583b8d5b8c30","Type":"ContainerStarted","Data":"bef6b370f4dc949905b860bac8f06d02cc2174631ef6be016314a279b36e933e"}
Jul 01 19:23:47.002710 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.002688    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a2b3861-694a-4821-bda3-2d6f27e7567a" containerID="d0f4243889a7808117604f022ade9863fada601e0a7128acb6cfec1744e5023f" exitCode=0
Jul 01 19:23:47.002794 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.002759    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerDied","Data":"d0f4243889a7808117604f022ade9863fada601e0a7128acb6cfec1744e5023f"}
Jul 01 19:23:47.004124 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.004104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq" event={"ID":"364edccc-1262-4d66-9533-ae4bd8a2c8b0","Type":"ContainerStarted","Data":"ebd43db9d460955ff84737eda10d6a7ddd1929dcd9844101dbf429d0b66dd4be"}
Jul 01 19:23:47.011931 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.011898    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rxhp8" podStartSLOduration=4.159898934 podStartE2EDuration="23.011887297s" totalImagesPullingTime="18.851988363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:47.011501403 +0000 UTC m=+23.560347778" watchObservedRunningTime="2026-07-01 19:23:47.011887297 +0000 UTC m=+23.560733672"
Jul 01 19:23:47.012218 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.012200    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dkmcn" podStartSLOduration=4.236881701 podStartE2EDuration="23.012195386s" totalImagesPullingTime="18.775313685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:46.022945814 +0000 UTC m=+22.571792190" watchObservedRunningTime="2026-07-01 19:23:47.012195386 +0000 UTC m=+23.561041793"
Jul 01 19:23:47.028796 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.028756    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9njfs" podStartSLOduration=3.703798991 podStartE2EDuration="23.028744863s" totalImagesPullingTime="19.324945872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:47.028492201 +0000 UTC m=+23.577338576" watchObservedRunningTime="2026-07-01 19:23:47.028744863 +0000 UTC m=+23.577591257"
Jul 01 19:23:47.042527 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.042491    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zzn2q" podStartSLOduration=4.285926507 podStartE2EDuration="23.042479388s" totalImagesPullingTime="18.756552881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:47.042176167 +0000 UTC m=+23.591022542" watchObservedRunningTime="2026-07-01 19:23:47.042479388 +0000 UTC m=+23.591325763"
Jul 01 19:23:47.056495 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.056464    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zxrk2" podStartSLOduration=4.213518324 podStartE2EDuration="23.056451854s" totalImagesPullingTime="18.84293353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:47.056402506 +0000 UTC m=+23.605248880" watchObservedRunningTime="2026-07-01 19:23:47.056451854 +0000 UTC m=+23.605298228"
Jul 01 19:23:47.376718 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.376689    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:47.377317 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.377301    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:47.447224 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.447198    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 19:23:47.913207 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.913168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:47.913499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.913168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:47.913499 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:47.913290    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:47.913499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.913168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:47.913499 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:47.913390    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:47.913499 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:47.913448    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:47.916609 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.916523    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T19:23:47.447220611Z","UUID":"baec8534-0359-4632-acc0-cc00d0eb5bbb","Handler":null,"Name":"","Endpoint":""}
Jul 01 19:23:47.918237 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.918206    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 19:23:47.918357 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:47.918244    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 19:23:48.007613 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:48.007577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hnm9j" event={"ID":"246949f7-3788-4e6f-8561-67b3ed764b7d","Type":"ContainerStarted","Data":"1ab53ff732723305e0329ee3b0642ed45fe4554dcc7923fa023d1446215f11aa"}
Jul 01 19:23:48.009871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:48.009842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq" event={"ID":"364edccc-1262-4d66-9533-ae4bd8a2c8b0","Type":"ContainerStarted","Data":"98eb5ac608ea58ab0d58e95548a684016aaa8669eff92acb4cd223630b31e0f3"}
Jul 01 19:23:48.434943 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:48.434900    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:48.435474 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:48.435452    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zzn2q"
Jul 01 19:23:48.451456 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:48.451411    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hnm9j" podStartSLOduration=5.602376336 podStartE2EDuration="24.451393383s" totalImagesPullingTime="18.849017047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:48.02590364 +0000 UTC m=+24.574750017" watchObservedRunningTime="2026-07-01 19:23:48.451393383 +0000 UTC m=+25.000239755"
Jul 01 19:23:49.015855 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:49.015824    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:23:49.016315 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:49.016285    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"08cac9bced23a1623d5b7040136baf672d06307423c1a197b01275d8c2890e7c"}
Jul 01 19:23:49.913031 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:49.912825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:49.913221 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:49.912861    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:49.913221 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:49.913139    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:49.913221 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:49.912962    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:49.913221 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:49.913212    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:49.913465 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:49.913329    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:51.913090 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:51.912792    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:51.913752 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:51.912795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:51.913752 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:51.913146    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:51.913752 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:51.912825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:51.913752 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:51.913272    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:51.913752 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:51.913208    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:52.025213 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.025175    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a2b3861-694a-4821-bda3-2d6f27e7567a" containerID="9fdaf35e481185b8e69371ca3b9e0ce15714431d901d88e869624f787ef098ce" exitCode=0
Jul 01 19:23:52.025395 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.025226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerDied","Data":"9fdaf35e481185b8e69371ca3b9e0ce15714431d901d88e869624f787ef098ce"}
Jul 01 19:23:52.027356 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.027330    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq" event={"ID":"364edccc-1262-4d66-9533-ae4bd8a2c8b0","Type":"ContainerStarted","Data":"ce9109b518a18dd20b54a3788fba0edac0f459e29dac2dc2f35d9445dfc177e4"}
Jul 01 19:23:52.030770 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.030750    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:23:52.031114 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.031093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"aea3ca8ce4b2d4241878d022b347b9e1eab3f969aa0a56850ca633782fcaa7ff"}
Jul 01 19:23:52.031418 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.031397    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:52.031505 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.031467    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:52.031555 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.031525    2566 scope.go:117] "RemoveContainer" containerID="7f7c26b3b6388787eb9a1e1924c2ea9f5a6b9787129865bec3b4942aab5a8497"
Jul 01 19:23:52.048588 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.048556    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:52.095042 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:52.094995    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgvbq" podStartSLOduration=4.184753732 podStartE2EDuration="28.094979509s" totalImagesPullingTime="23.910225777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:52.094529028 +0000 UTC m=+28.643375406" watchObservedRunningTime="2026-07-01 19:23:52.094979509 +0000 UTC m=+28.643825884"
Jul 01 19:23:53.037100 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.037071    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:23:53.037564 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.037518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" event={"ID":"8dc0ae12-7f88-49fe-8193-2c103f614edb","Type":"ContainerStarted","Data":"9a5b4f75a6dbccfc56a2fe2b0a91d7530c6c74175abde811efc5ed3f4f5954bb"}
Jul 01 19:23:53.038265 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.037905    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:53.055222 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.055183    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:23:53.073819 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.073765    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj" podStartSLOduration=10.191917277 podStartE2EDuration="29.073748739s" totalImagesPullingTime="18.881831462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:53.068128128 +0000 UTC m=+29.616974504" watchObservedRunningTime="2026-07-01 19:23:53.073748739 +0000 UTC m=+29.622595111"
Jul 01 19:23:53.324406 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.324219    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4wtck"]
Jul 01 19:23:53.324581 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.324555    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:53.324689 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:53.324667    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:53.327175 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.327148    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bdsqr"]
Jul 01 19:23:53.327298 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.327270    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:53.327415 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:53.327386    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:53.327846 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.327822    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qw5hn"]
Jul 01 19:23:53.327955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:53.327941    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:53.328069 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:53.328048    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:54.041269 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:54.041230    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a2b3861-694a-4821-bda3-2d6f27e7567a" containerID="ed0e7a90f768443adc2c2d7a404afb999738cbbe06e092f52837dbf81a2e6ad3" exitCode=0
Jul 01 19:23:54.041859 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:54.041313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerDied","Data":"ed0e7a90f768443adc2c2d7a404afb999738cbbe06e092f52837dbf81a2e6ad3"}
Jul 01 19:23:54.912939 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:54.912906    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:54.913098 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:54.913003    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:54.913098 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:54.913009    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:54.913098 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:54.913032    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:54.913215 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:54.913111    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:54.913215 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:54.913169    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:56.047935 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:56.047904    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a2b3861-694a-4821-bda3-2d6f27e7567a" containerID="0cdf49322264a90187ef0509c685912e93ed01d2a07b8e14b784c673af746bd8" exitCode=0
Jul 01 19:23:56.048596 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:56.047965    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerDied","Data":"0cdf49322264a90187ef0509c685912e93ed01d2a07b8e14b784c673af746bd8"}
Jul 01 19:23:56.913587 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:56.913548    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:56.913753 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:56.913590    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:56.913753 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:56.913601    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:56.913753 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:56.913676    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5hn" podUID="aed27bed-2de0-4745-8ff1-b0c45f6a90ef"
Jul 01 19:23:56.913753 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:56.913743    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdsqr" podUID="017149d2-7907-4bb3-8a16-780616d86470"
Jul 01 19:23:56.913895 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:56.913807    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wtck" podUID="ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d"
Jul 01 19:23:57.593378 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:57.593262    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:57.593378 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:57.593323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:57.593886 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.593450    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:57.593886 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.593452    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:57.593886 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.593515    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs podName:aed27bed-2de0-4745-8ff1-b0c45f6a90ef nodeName:}" failed. No retries permitted until 2026-07-01 19:24:29.593498119 +0000 UTC m=+66.142344477 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs") pod "network-metrics-daemon-qw5hn" (UID: "aed27bed-2de0-4745-8ff1-b0c45f6a90ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:23:57.593886 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.593535    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret podName:ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d nodeName:}" failed. No retries permitted until 2026-07-01 19:24:29.593525036 +0000 UTC m=+66.142371389 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret") pod "global-pull-secret-syncer-4wtck" (UID: "ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:23:57.794858 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:57.794827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:57.795039 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.794978    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:23:57.795039 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.795000    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:23:57.795039 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.795011    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jxfz for pod openshift-network-diagnostics/network-check-target-bdsqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:57.795181 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:57.795078    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz podName:017149d2-7907-4bb3-8a16-780616d86470 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:29.795059217 +0000 UTC m=+66.343905592 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxfz" (UniqueName: "kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz") pod "network-check-target-bdsqr" (UID: "017149d2-7907-4bb3-8a16-780616d86470") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:23:58.783176 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.783100    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeReady"
Jul 01 19:23:58.783667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.783242    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 19:23:58.816874 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.816841    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f88cc5f4-684dq"]
Jul 01 19:23:58.819424 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.819382    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.821675 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.821649    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 19:23:58.821798 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.821727    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 19:23:58.821956 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.821937    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 19:23:58.822028 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.821970    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nzzx8\""
Jul 01 19:23:58.826484 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.826461    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 19:23:58.827741 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.827502    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-646vp"]
Jul 01 19:23:58.829185 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.829165    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l7bpz"]
Jul 01 19:23:58.829316 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.829295    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:58.831151 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.831131    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:58.831620 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.831600    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 19:23:58.831723 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.831668    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 19:23:58.831786 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.831764    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 19:23:58.831896 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.831882    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qjjt5\""
Jul 01 19:23:58.833248 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.833225    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f88cc5f4-684dq"]
Jul 01 19:23:58.833447 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.833423    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 19:23:58.833535 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.833448    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9p2w7\""
Jul 01 19:23:58.833687 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.833588    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 19:23:58.839652 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.839592    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-646vp"]
Jul 01 19:23:58.843396 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.842462    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7bpz"]
Jul 01 19:23:58.850383 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.850341    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hwgxh"]
Jul 01 19:23:58.852342 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.852324    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:58.854812 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.854789    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 19:23:58.854923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.854791    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 19:23:58.854923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.854875    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 19:23:58.855095 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.854983    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 19:23:58.855302 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.855283    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5hd6c\""
Jul 01 19:23:58.865527 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.865501    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hwgxh"]
Jul 01 19:23:58.902942 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.902911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/998ccf1d-3c43-480e-8433-b4cb2cc2123a-config-volume\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:58.903117 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.902957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:58.903117 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxs22\" (UniqueName: \"kubernetes.io/projected/3bf8d620-9462-42bb-a12b-8204bd79abab-kube-api-access-fxs22\") pod \"ingress-canary-646vp\" (UID: \"3bf8d620-9462-42bb-a12b-8204bd79abab\") " pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:58.903117 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903034    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/998ccf1d-3c43-480e-8433-b4cb2cc2123a-tmp-dir\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:58.903117 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903055    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8v7v\" (UniqueName: \"kubernetes.io/projected/998ccf1d-3c43-480e-8433-b4cb2cc2123a-kube-api-access-c8v7v\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:58.903117 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-bound-sa-token\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djhng\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-kube-api-access-djhng\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903190    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92c4f69c-06a5-46aa-abd0-0eca63ca7440-ca-trust-extracted\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903250    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf8d620-9462-42bb-a12b-8204bd79abab-cert\") pod \"ingress-canary-646vp\" (UID: \"3bf8d620-9462-42bb-a12b-8204bd79abab\") " pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:58.903309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903267    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-image-registry-private-configuration\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903292    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-certificates\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903568 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903395    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-installation-pull-secrets\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.903568 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.903463    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-trusted-ca\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:58.912922 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.912895    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:23:58.913093 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.913016    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:23:58.913093 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.913072    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:23:58.915667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.915640    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-64bmf\""
Jul 01 19:23:58.915667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.915654    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 19:23:58.915667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.915669    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 19:23:58.915918 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.915705    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 19:23:58.915918 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.915659    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 19:23:58.916079 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:58.916064    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4nv7g\""
Jul 01 19:23:59.003909 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.003866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-bound-sa-token\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004107 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.003920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djhng\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-kube-api-access-djhng\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004107 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.004107 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004071    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mq2p\" (UniqueName: \"kubernetes.io/projected/43fa70b6-35bc-49e9-83ba-44aed748412c-kube-api-access-9mq2p\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.004107 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004302 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92c4f69c-06a5-46aa-abd0-0eca63ca7440-ca-trust-extracted\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004302 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004210    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf8d620-9462-42bb-a12b-8204bd79abab-cert\") pod \"ingress-canary-646vp\" (UID: \"3bf8d620-9462-42bb-a12b-8204bd79abab\") " pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:59.004302 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-image-registry-private-configuration\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004302 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-certificates\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004305    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-installation-pull-secrets\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-trusted-ca\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004452    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/43fa70b6-35bc-49e9-83ba-44aed748412c-data-volume\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.004499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/43fa70b6-35bc-49e9-83ba-44aed748412c-crio-socket\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/998ccf1d-3c43-480e-8433-b4cb2cc2123a-config-volume\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004588    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43fa70b6-35bc-49e9-83ba-44aed748412c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004590    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92c4f69c-06a5-46aa-abd0-0eca63ca7440-ca-trust-extracted\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.004182    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.004632    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f88cc5f4-684dq: secret "image-registry-tls" not found
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxs22\" (UniqueName: \"kubernetes.io/projected/3bf8d620-9462-42bb-a12b-8204bd79abab-kube-api-access-fxs22\") pod \"ingress-canary-646vp\" (UID: \"3bf8d620-9462-42bb-a12b-8204bd79abab\") " pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/998ccf1d-3c43-480e-8433-b4cb2cc2123a-tmp-dir\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.004680 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.004683    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls podName:92c4f69c-06a5-46aa-abd0-0eca63ca7440 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:59.504664017 +0000 UTC m=+36.053510369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls") pod "image-registry-6f88cc5f4-684dq" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440") : secret "image-registry-tls" not found
Jul 01 19:23:59.005115 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.004709    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8v7v\" (UniqueName: \"kubernetes.io/projected/998ccf1d-3c43-480e-8433-b4cb2cc2123a-kube-api-access-c8v7v\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.005115 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.005010    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/998ccf1d-3c43-480e-8433-b4cb2cc2123a-tmp-dir\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.005214 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.005133    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:23:59.005214 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.005212    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls podName:998ccf1d-3c43-480e-8433-b4cb2cc2123a nodeName:}" failed. No retries permitted until 2026-07-01 19:23:59.505192593 +0000 UTC m=+36.054038958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls") pod "dns-default-l7bpz" (UID: "998ccf1d-3c43-480e-8433-b4cb2cc2123a") : secret "dns-default-metrics-tls" not found
Jul 01 19:23:59.005796 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.005770    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-certificates\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.005907 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.005810    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/998ccf1d-3c43-480e-8433-b4cb2cc2123a-config-volume\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.005970 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.005952    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-trusted-ca\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.008891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.008866    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-image-registry-private-configuration\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.008891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.008878    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf8d620-9462-42bb-a12b-8204bd79abab-cert\") pod \"ingress-canary-646vp\" (UID: \"3bf8d620-9462-42bb-a12b-8204bd79abab\") " pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:59.009039 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.008917    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-installation-pull-secrets\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.016563 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.016529    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8v7v\" (UniqueName: \"kubernetes.io/projected/998ccf1d-3c43-480e-8433-b4cb2cc2123a-kube-api-access-c8v7v\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.017559 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.017500    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djhng\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-kube-api-access-djhng\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.017559 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.017498    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxs22\" (UniqueName: \"kubernetes.io/projected/3bf8d620-9462-42bb-a12b-8204bd79abab-kube-api-access-fxs22\") pod \"ingress-canary-646vp\" (UID: \"3bf8d620-9462-42bb-a12b-8204bd79abab\") " pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:59.018288 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.018264    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-bound-sa-token\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.105280 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/43fa70b6-35bc-49e9-83ba-44aed748412c-crio-socket\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.105493 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43fa70b6-35bc-49e9-83ba-44aed748412c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.105493 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.105493 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mq2p\" (UniqueName: \"kubernetes.io/projected/43fa70b6-35bc-49e9-83ba-44aed748412c-kube-api-access-9mq2p\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.105646 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105499    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/43fa70b6-35bc-49e9-83ba-44aed748412c-data-volume\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.105646 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105506    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/43fa70b6-35bc-49e9-83ba-44aed748412c-crio-socket\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.105646 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.105524    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:23:59.105646 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.105613    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls podName:43fa70b6-35bc-49e9-83ba-44aed748412c nodeName:}" failed. No retries permitted until 2026-07-01 19:23:59.605591393 +0000 UTC m=+36.154437765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwgxh" (UID: "43fa70b6-35bc-49e9-83ba-44aed748412c") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:23:59.105852 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.105828    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/43fa70b6-35bc-49e9-83ba-44aed748412c-data-volume\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.106043 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.106022    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43fa70b6-35bc-49e9-83ba-44aed748412c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.116412 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.116349    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mq2p\" (UniqueName: \"kubernetes.io/projected/43fa70b6-35bc-49e9-83ba-44aed748412c-kube-api-access-9mq2p\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.141449 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.141417    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-646vp"
Jul 01 19:23:59.493759 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.493520    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"]
Jul 01 19:23:59.495653 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.495631    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.498913 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.498890    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 01 19:23:59.500755 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.500734    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 01 19:23:59.500871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.500785    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 01 19:23:59.501695 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.501677    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qp5lr\""
Jul 01 19:23:59.502871 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.502851    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 01 19:23:59.509590 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.509568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:23:59.509695 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.509664    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:23:59.509755 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.509708    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:23:59.509755 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.509724    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f88cc5f4-684dq: secret "image-registry-tls" not found
Jul 01 19:23:59.509887 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.509782    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls podName:92c4f69c-06a5-46aa-abd0-0eca63ca7440 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:00.509761378 +0000 UTC m=+37.058607743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls") pod "image-registry-6f88cc5f4-684dq" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440") : secret "image-registry-tls" not found
Jul 01 19:23:59.509887 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.509794    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:23:59.509887 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.509852    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls podName:998ccf1d-3c43-480e-8433-b4cb2cc2123a nodeName:}" failed. No retries permitted until 2026-07-01 19:24:00.509835141 +0000 UTC m=+37.058681500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls") pod "dns-default-l7bpz" (UID: "998ccf1d-3c43-480e-8433-b4cb2cc2123a") : secret "dns-default-metrics-tls" not found
Jul 01 19:23:59.523268 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.523234    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"]
Jul 01 19:23:59.611104 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.611060    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:23:59.611298 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.611118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn58z\" (UniqueName: \"kubernetes.io/projected/9bb538d3-454f-43c9-a846-1254ebeabaac-kube-api-access-kn58z\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.611298 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.611156    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9bb538d3-454f-43c9-a846-1254ebeabaac-signing-key\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.611298 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.611177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9bb538d3-454f-43c9-a846-1254ebeabaac-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.611298 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.611220    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:23:59.611298 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:23:59.611280    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls podName:43fa70b6-35bc-49e9-83ba-44aed748412c nodeName:}" failed. No retries permitted until 2026-07-01 19:24:00.611262078 +0000 UTC m=+37.160108433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwgxh" (UID: "43fa70b6-35bc-49e9-83ba-44aed748412c") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:23:59.711934 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.711893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn58z\" (UniqueName: \"kubernetes.io/projected/9bb538d3-454f-43c9-a846-1254ebeabaac-kube-api-access-kn58z\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.712112 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.711951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9bb538d3-454f-43c9-a846-1254ebeabaac-signing-key\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.712112 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.711980    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9bb538d3-454f-43c9-a846-1254ebeabaac-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.712727 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.712671    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9bb538d3-454f-43c9-a846-1254ebeabaac-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.715161 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.715139    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9bb538d3-454f-43c9-a846-1254ebeabaac-signing-key\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.720569 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.720547    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn58z\" (UniqueName: \"kubernetes.io/projected/9bb538d3-454f-43c9-a846-1254ebeabaac-kube-api-access-kn58z\") pod \"service-ca-7bb4f4bcc7-7fvh4\" (UID: \"9bb538d3-454f-43c9-a846-1254ebeabaac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:23:59.806910 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:23:59.806880    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"
Jul 01 19:24:00.069812 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:00.069708    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dkmcn_0e22d576-1039-4788-a13d-a98075ea84b4/dns-node-resolver/0.log"
Jul 01 19:24:00.117121 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:00.117086    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-646vp"]
Jul 01 19:24:00.117955 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:00.117932    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4"]
Jul 01 19:24:00.121093 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:00.121069    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf8d620_9462_42bb_a12b_8204bd79abab.slice/crio-6787a803d4ec43f54ada498fb54f0883f9bd370944ae4d4c2534d6f70cb62761 WatchSource:0}: Error finding container 6787a803d4ec43f54ada498fb54f0883f9bd370944ae4d4c2534d6f70cb62761: Status 404 returned error can't find the container with id 6787a803d4ec43f54ada498fb54f0883f9bd370944ae4d4c2534d6f70cb62761
Jul 01 19:24:00.122210 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:00.122188    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb538d3_454f_43c9_a846_1254ebeabaac.slice/crio-4e2592f3a508dd0a7c19f65e29234c6efc9ad401eb78ea18e7640417a8cc0592 WatchSource:0}: Error finding container 4e2592f3a508dd0a7c19f65e29234c6efc9ad401eb78ea18e7640417a8cc0592: Status 404 returned error can't find the container with id 4e2592f3a508dd0a7c19f65e29234c6efc9ad401eb78ea18e7640417a8cc0592
Jul 01 19:24:00.521259 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:00.521171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:00.521259 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:00.521234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:00.521462 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.521325    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:24:00.521462 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.521403    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls podName:998ccf1d-3c43-480e-8433-b4cb2cc2123a nodeName:}" failed. No retries permitted until 2026-07-01 19:24:02.521388331 +0000 UTC m=+39.070234689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls") pod "dns-default-l7bpz" (UID: "998ccf1d-3c43-480e-8433-b4cb2cc2123a") : secret "dns-default-metrics-tls" not found
Jul 01 19:24:00.521462 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.521330    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:24:00.521462 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.521451    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f88cc5f4-684dq: secret "image-registry-tls" not found
Jul 01 19:24:00.521588 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.521493    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls podName:92c4f69c-06a5-46aa-abd0-0eca63ca7440 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:02.521482605 +0000 UTC m=+39.070328962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls") pod "image-registry-6f88cc5f4-684dq" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440") : secret "image-registry-tls" not found
Jul 01 19:24:00.622120 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:00.622081    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:24:00.622277 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.622226    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:24:00.622316 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:00.622290    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls podName:43fa70b6-35bc-49e9-83ba-44aed748412c nodeName:}" failed. No retries permitted until 2026-07-01 19:24:02.622274326 +0000 UTC m=+39.171120682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwgxh" (UID: "43fa70b6-35bc-49e9-83ba-44aed748412c") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:24:01.064318 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:01.064281    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a2b3861-694a-4821-bda3-2d6f27e7567a" containerID="1e159bce62f52a7c005dcbb8a0240463e0f65723cd3d946f6e83fcef8a768aeb" exitCode=0
Jul 01 19:24:01.064811 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:01.064409    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerDied","Data":"1e159bce62f52a7c005dcbb8a0240463e0f65723cd3d946f6e83fcef8a768aeb"}
Jul 01 19:24:01.066888 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:01.066855    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-646vp" event={"ID":"3bf8d620-9462-42bb-a12b-8204bd79abab","Type":"ContainerStarted","Data":"6787a803d4ec43f54ada498fb54f0883f9bd370944ae4d4c2534d6f70cb62761"}
Jul 01 19:24:01.069570 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:01.069505    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4" event={"ID":"9bb538d3-454f-43c9-a846-1254ebeabaac","Type":"ContainerStarted","Data":"4e2592f3a508dd0a7c19f65e29234c6efc9ad401eb78ea18e7640417a8cc0592"}
Jul 01 19:24:01.250061 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:01.250034    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zxrk2_193f723e-55a0-49d9-9f96-d222090fb79d/node-ca/0.log"
Jul 01 19:24:02.075664 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:02.075625    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a2b3861-694a-4821-bda3-2d6f27e7567a" containerID="93e3e5d1cf59c63135340549e01732e6060d8fe54d217d642afd731b238d694e" exitCode=0
Jul 01 19:24:02.076155 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:02.075688    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerDied","Data":"93e3e5d1cf59c63135340549e01732e6060d8fe54d217d642afd731b238d694e"}
Jul 01 19:24:02.540081 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:02.540045    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:02.540248 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:02.540140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:02.540248 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.540221    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:24:02.540248 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.540241    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f88cc5f4-684dq: secret "image-registry-tls" not found
Jul 01 19:24:02.540398 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.540297    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls podName:92c4f69c-06a5-46aa-abd0-0eca63ca7440 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:06.540282223 +0000 UTC m=+43.089128579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls") pod "image-registry-6f88cc5f4-684dq" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440") : secret "image-registry-tls" not found
Jul 01 19:24:02.540398 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.540309    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:24:02.540398 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.540390    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls podName:998ccf1d-3c43-480e-8433-b4cb2cc2123a nodeName:}" failed. No retries permitted until 2026-07-01 19:24:06.540355796 +0000 UTC m=+43.089202162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls") pod "dns-default-l7bpz" (UID: "998ccf1d-3c43-480e-8433-b4cb2cc2123a") : secret "dns-default-metrics-tls" not found
Jul 01 19:24:02.641454 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:02.641413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:24:02.641643 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.641581    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:24:02.641701 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:02.641666    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls podName:43fa70b6-35bc-49e9-83ba-44aed748412c nodeName:}" failed. No retries permitted until 2026-07-01 19:24:06.641642853 +0000 UTC m=+43.190489205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwgxh" (UID: "43fa70b6-35bc-49e9-83ba-44aed748412c") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:24:04.084583 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:04.084536    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" event={"ID":"2a2b3861-694a-4821-bda3-2d6f27e7567a","Type":"ContainerStarted","Data":"72a5f0f065eb735d68aa6c0c430bdd6f0bf4e2c79687a7b275f9858acda8011f"}
Jul 01 19:24:04.086068 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:04.086043    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-646vp" event={"ID":"3bf8d620-9462-42bb-a12b-8204bd79abab","Type":"ContainerStarted","Data":"0a29358625c985bd1bf9dedfc6d95bff2dcfb398a8f7fdc443d06ff11ce4fccc"}
Jul 01 19:24:04.087419 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:04.087397    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4" event={"ID":"9bb538d3-454f-43c9-a846-1254ebeabaac","Type":"ContainerStarted","Data":"7d3298f3887ff9d6f20b629c087c92f9a5173728b45238ab0741a2653e35bc33"}
Jul 01 19:24:04.155710 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:04.155653    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zmrdr" podStartSLOduration=10.258426389 podStartE2EDuration="40.155638517s" totalImagesPullingTime="29.897212128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:04.152517924 +0000 UTC m=+40.701364298" watchObservedRunningTime="2026-07-01 19:24:04.155638517 +0000 UTC m=+40.704484891"
Jul 01 19:24:04.184513 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:04.184459    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-646vp" podStartSLOduration=2.9164510359999998 podStartE2EDuration="6.184442004s" totalImagesPullingTime="3.267990968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:04.183928133 +0000 UTC m=+40.732774508" watchObservedRunningTime="2026-07-01 19:24:04.184442004 +0000 UTC m=+40.733288376"
Jul 01 19:24:04.222937 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:04.222873    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-7fvh4" podStartSLOduration=1.96169813 podStartE2EDuration="5.222855968s" totalImagesPullingTime="3.261157838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:04.222195017 +0000 UTC m=+40.771041392" watchObservedRunningTime="2026-07-01 19:24:04.222855968 +0000 UTC m=+40.771702342"
Jul 01 19:24:06.575072 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:06.575031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:06.575467 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:06.575093    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:06.575467 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.575196    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:24:06.575467 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.575196    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:24:06.575467 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.575259    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls podName:998ccf1d-3c43-480e-8433-b4cb2cc2123a nodeName:}" failed. No retries permitted until 2026-07-01 19:24:14.575244214 +0000 UTC m=+51.124090571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls") pod "dns-default-l7bpz" (UID: "998ccf1d-3c43-480e-8433-b4cb2cc2123a") : secret "dns-default-metrics-tls" not found
Jul 01 19:24:06.575467 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.575206    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f88cc5f4-684dq: secret "image-registry-tls" not found
Jul 01 19:24:06.575467 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.575324    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls podName:92c4f69c-06a5-46aa-abd0-0eca63ca7440 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:14.575313191 +0000 UTC m=+51.124159544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls") pod "image-registry-6f88cc5f4-684dq" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440") : secret "image-registry-tls" not found
Jul 01 19:24:06.676094 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:06.676052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:24:06.676241 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.676195    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:24:06.676300 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:06.676261    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls podName:43fa70b6-35bc-49e9-83ba-44aed748412c nodeName:}" failed. No retries permitted until 2026-07-01 19:24:14.676246061 +0000 UTC m=+51.225092413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwgxh" (UID: "43fa70b6-35bc-49e9-83ba-44aed748412c") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:24:14.644261 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.644219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:14.644858 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.644298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:14.648161 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.648131    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/998ccf1d-3c43-480e-8433-b4cb2cc2123a-metrics-tls\") pod \"dns-default-l7bpz\" (UID: \"998ccf1d-3c43-480e-8433-b4cb2cc2123a\") " pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:14.648282 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.648160    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"image-registry-6f88cc5f4-684dq\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") " pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:14.731344 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.731306    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:14.745646 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.745617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:24:14.748150 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.748077    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43fa70b6-35bc-49e9-83ba-44aed748412c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwgxh\" (UID: \"43fa70b6-35bc-49e9-83ba-44aed748412c\") " pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:24:14.751029 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.751007    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:14.762158 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.762138    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hwgxh"
Jul 01 19:24:14.917801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.917729    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f88cc5f4-684dq"]
Jul 01 19:24:14.922488 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:14.922459    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c4f69c_06a5_46aa_abd0_0eca63ca7440.slice/crio-71c362620c82ccd2e3985a815c2c105d706a9668a4a798b60f78f118e1d63c05 WatchSource:0}: Error finding container 71c362620c82ccd2e3985a815c2c105d706a9668a4a798b60f78f118e1d63c05: Status 404 returned error can't find the container with id 71c362620c82ccd2e3985a815c2c105d706a9668a4a798b60f78f118e1d63c05
Jul 01 19:24:14.937703 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.937678    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7bpz"]
Jul 01 19:24:14.940812 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:14.940788    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998ccf1d_3c43_480e_8433_b4cb2cc2123a.slice/crio-34dbcb8298b7f3f0449a78ce82a364dae294a040539309e07fe2ecc00f913f57 WatchSource:0}: Error finding container 34dbcb8298b7f3f0449a78ce82a364dae294a040539309e07fe2ecc00f913f57: Status 404 returned error can't find the container with id 34dbcb8298b7f3f0449a78ce82a364dae294a040539309e07fe2ecc00f913f57
Jul 01 19:24:14.954457 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:14.954436    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hwgxh"]
Jul 01 19:24:14.957266 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:14.957249    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fa70b6_35bc_49e9_83ba_44aed748412c.slice/crio-bd27489e1be49faef8ec6cbf76efcd7faacef082f3e642f7dae158642301ab13 WatchSource:0}: Error finding container bd27489e1be49faef8ec6cbf76efcd7faacef082f3e642f7dae158642301ab13: Status 404 returned error can't find the container with id bd27489e1be49faef8ec6cbf76efcd7faacef082f3e642f7dae158642301ab13
Jul 01 19:24:15.117061 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:15.117021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7bpz" event={"ID":"998ccf1d-3c43-480e-8433-b4cb2cc2123a","Type":"ContainerStarted","Data":"34dbcb8298b7f3f0449a78ce82a364dae294a040539309e07fe2ecc00f913f57"}
Jul 01 19:24:15.118506 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:15.118478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" event={"ID":"92c4f69c-06a5-46aa-abd0-0eca63ca7440","Type":"ContainerStarted","Data":"85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e"}
Jul 01 19:24:15.118624 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:15.118518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" event={"ID":"92c4f69c-06a5-46aa-abd0-0eca63ca7440","Type":"ContainerStarted","Data":"71c362620c82ccd2e3985a815c2c105d706a9668a4a798b60f78f118e1d63c05"}
Jul 01 19:24:15.118624 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:15.118605    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:15.119692 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:15.119669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwgxh" event={"ID":"43fa70b6-35bc-49e9-83ba-44aed748412c","Type":"ContainerStarted","Data":"bd27489e1be49faef8ec6cbf76efcd7faacef082f3e642f7dae158642301ab13"}
Jul 01 19:24:15.155544 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:15.155471    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" podStartSLOduration=45.155458286 podStartE2EDuration="45.155458286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:15.154102067 +0000 UTC m=+51.702948456" watchObservedRunningTime="2026-07-01 19:24:15.155458286 +0000 UTC m=+51.704304660"
Jul 01 19:24:20.136138 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:20.136110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7bpz" event={"ID":"998ccf1d-3c43-480e-8433-b4cb2cc2123a","Type":"ContainerStarted","Data":"01e721145badaef093352ca5348114a81919196433fd0508d05cdeacc76be30e"}
Jul 01 19:24:20.136468 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:20.136162    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7bpz" event={"ID":"998ccf1d-3c43-480e-8433-b4cb2cc2123a","Type":"ContainerStarted","Data":"a9f1383c1d18d0753458c506b412d20881dde1becdca2580a0e0a6cddc665519"}
Jul 01 19:24:20.136468 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:20.136231    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:20.137739 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:20.137722    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwgxh" event={"ID":"43fa70b6-35bc-49e9-83ba-44aed748412c","Type":"ContainerStarted","Data":"cbc9fac61555e9240786d0dae0c6125a88d108b6ce6dee518d910edea5f0df4e"}
Jul 01 19:24:20.153082 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:20.153027    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l7bpz" podStartSLOduration=17.759124837 podStartE2EDuration="22.153009119s" totalImagesPullingTime="4.393884282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:20.152024279 +0000 UTC m=+56.700870653" watchObservedRunningTime="2026-07-01 19:24:20.153009119 +0000 UTC m=+56.701855494"
Jul 01 19:24:21.142602 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.142562    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwgxh" event={"ID":"43fa70b6-35bc-49e9-83ba-44aed748412c","Type":"ContainerStarted","Data":"7000af88437d67f96fe838d0c63840404d99e65c34015498b5bbdee6bcb2ea70"}
Jul 01 19:24:21.341937 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.341891    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"]
Jul 01 19:24:21.343836 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.343816    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"]
Jul 01 19:24:21.343976 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.343961    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.345685 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.345667    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.346496 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.346474    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 01 19:24:21.346997 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.346977    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 01 19:24:21.347455 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.347435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 01 19:24:21.347639 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.347438    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 01 19:24:21.349745 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.349724    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"]
Jul 01 19:24:21.350519 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.350497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fqzfd\""
Jul 01 19:24:21.350623 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.350504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 01 19:24:21.351757 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.351741    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.354088 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.353855    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 01 19:24:21.354088 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.353944    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 01 19:24:21.354088 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.354015    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 01 19:24:21.354263 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.354155    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 01 19:24:21.358909 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.358877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"]
Jul 01 19:24:21.359588 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.359570    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"]
Jul 01 19:24:21.368643 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.368621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"]
Jul 01 19:24:21.377807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.377785    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f88cc5f4-684dq"]
Jul 01 19:24:21.399179 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53dde71e-6f80-4d73-b4b1-8f3395466d98-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-746c5446db-h5rqh\" (UID: \"53dde71e-6f80-4d73-b4b1-8f3395466d98\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.399179 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.399391 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399251    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snm65\" (UniqueName: \"kubernetes.io/projected/c42e0814-fe61-43ab-a277-0804604a29de-kube-api-access-snm65\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.399391 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399288    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c42e0814-fe61-43ab-a277-0804604a29de-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.399507 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399361    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlzt2\" (UniqueName: \"kubernetes.io/projected/53dde71e-6f80-4d73-b4b1-8f3395466d98-kube-api-access-hlzt2\") pod \"managed-serviceaccount-addon-agent-746c5446db-h5rqh\" (UID: \"53dde71e-6f80-4d73-b4b1-8f3395466d98\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.399563 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c42e0814-fe61-43ab-a277-0804604a29de-tmp\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.399619 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399582    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.399671 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399618    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.399671 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9frf\" (UniqueName: \"kubernetes.io/projected/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-kube-api-access-t9frf\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.399772 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-hub\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.399772 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.399740    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-ca\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.439033 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.439000    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dcb499546-hckkd"]
Jul 01 19:24:21.441188 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.441152    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.460411 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.458508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dcb499546-hckkd"]
Jul 01 19:24:21.501127 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snm65\" (UniqueName: \"kubernetes.io/projected/c42e0814-fe61-43ab-a277-0804604a29de-kube-api-access-snm65\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501141    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c42e0814-fe61-43ab-a277-0804604a29de-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlzt2\" (UniqueName: \"kubernetes.io/projected/53dde71e-6f80-4d73-b4b1-8f3395466d98-kube-api-access-hlzt2\") pod \"managed-serviceaccount-addon-agent-746c5446db-h5rqh\" (UID: \"53dde71e-6f80-4d73-b4b1-8f3395466d98\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-bound-sa-token\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501251    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c42e0814-fe61-43ab-a277-0804604a29de-tmp\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501282    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501308    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21836a16-868d-46fb-93e8-5da6f6b471f7-installation-pull-secrets\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-registry-tls\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501398 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21836a16-868d-46fb-93e8-5da6f6b471f7-registry-certificates\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9frf\" (UniqueName: \"kubernetes.io/projected/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-kube-api-access-t9frf\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgjhv\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-kube-api-access-hgjhv\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501512    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21836a16-868d-46fb-93e8-5da6f6b471f7-image-registry-private-configuration\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21836a16-868d-46fb-93e8-5da6f6b471f7-ca-trust-extracted\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-hub\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-ca\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501633    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53dde71e-6f80-4d73-b4b1-8f3395466d98-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-746c5446db-h5rqh\" (UID: \"53dde71e-6f80-4d73-b4b1-8f3395466d98\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21836a16-868d-46fb-93e8-5da6f6b471f7-trusted-ca\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.501746 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.502276 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.501950    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c42e0814-fe61-43ab-a277-0804604a29de-tmp\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.502460 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.502436    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.504890 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.504859    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c42e0814-fe61-43ab-a277-0804604a29de-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.505022 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.504999    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.505348 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.505325    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.505461 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.505347    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53dde71e-6f80-4d73-b4b1-8f3395466d98-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-746c5446db-h5rqh\" (UID: \"53dde71e-6f80-4d73-b4b1-8f3395466d98\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.505845 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.505826    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-hub\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.506200 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.506174    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-ca\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.513027 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.512995    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9frf\" (UniqueName: \"kubernetes.io/projected/74d7549c-4bc1-4749-ba0d-ac34e7d9457a-kube-api-access-t9frf\") pod \"cluster-proxy-proxy-agent-766fbbd9d-57hs4\" (UID: \"74d7549c-4bc1-4749-ba0d-ac34e7d9457a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.513433 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.513388    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snm65\" (UniqueName: \"kubernetes.io/projected/c42e0814-fe61-43ab-a277-0804604a29de-kube-api-access-snm65\") pod \"klusterlet-addon-workmgr-5bf669d4c8-c6748\" (UID: \"c42e0814-fe61-43ab-a277-0804604a29de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.514975 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.514939    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlzt2\" (UniqueName: \"kubernetes.io/projected/53dde71e-6f80-4d73-b4b1-8f3395466d98-kube-api-access-hlzt2\") pod \"managed-serviceaccount-addon-agent-746c5446db-h5rqh\" (UID: \"53dde71e-6f80-4d73-b4b1-8f3395466d98\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.603057 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603013    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-bound-sa-token\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603231 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21836a16-868d-46fb-93e8-5da6f6b471f7-installation-pull-secrets\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603231 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-registry-tls\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603231 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21836a16-868d-46fb-93e8-5da6f6b471f7-registry-certificates\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603231 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgjhv\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-kube-api-access-hgjhv\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603231 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21836a16-868d-46fb-93e8-5da6f6b471f7-image-registry-private-configuration\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603813 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21836a16-868d-46fb-93e8-5da6f6b471f7-ca-trust-extracted\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603813 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21836a16-868d-46fb-93e8-5da6f6b471f7-trusted-ca\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.603920 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.603898    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21836a16-868d-46fb-93e8-5da6f6b471f7-ca-trust-extracted\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.604397 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.604319    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21836a16-868d-46fb-93e8-5da6f6b471f7-trusted-ca\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.604512 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.604385    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21836a16-868d-46fb-93e8-5da6f6b471f7-registry-certificates\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.606631 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.606608    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21836a16-868d-46fb-93e8-5da6f6b471f7-image-registry-private-configuration\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.606631 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.606620    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21836a16-868d-46fb-93e8-5da6f6b471f7-installation-pull-secrets\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.607083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.607060    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-registry-tls\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.613804 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.613783    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-bound-sa-token\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.614145 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.614127    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgjhv\" (UniqueName: \"kubernetes.io/projected/21836a16-868d-46fb-93e8-5da6f6b471f7-kube-api-access-hgjhv\") pod \"image-registry-7dcb499546-hckkd\" (UID: \"21836a16-868d-46fb-93e8-5da6f6b471f7\") " pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.659636 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.659542    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:21.684524 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.684491    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"
Jul 01 19:24:21.691036 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.691009    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"
Jul 01 19:24:21.764006 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.763680    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:21.832420 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.832385    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"]
Jul 01 19:24:21.839015 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:21.838983    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42e0814_fe61_43ab_a277_0804604a29de.slice/crio-ba254c9f031444998258d4e7dafbd40b08f265bba504a97bd51c5092f03a1d6f WatchSource:0}: Error finding container ba254c9f031444998258d4e7dafbd40b08f265bba504a97bd51c5092f03a1d6f: Status 404 returned error can't find the container with id ba254c9f031444998258d4e7dafbd40b08f265bba504a97bd51c5092f03a1d6f
Jul 01 19:24:21.849850 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.848711    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4"]
Jul 01 19:24:21.854338 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:21.854306    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74d7549c_4bc1_4749_ba0d_ac34e7d9457a.slice/crio-a997068930e6b3ef5a96e097d3d494f43fe0131664513de4666ef1f33dd23df1 WatchSource:0}: Error finding container a997068930e6b3ef5a96e097d3d494f43fe0131664513de4666ef1f33dd23df1: Status 404 returned error can't find the container with id a997068930e6b3ef5a96e097d3d494f43fe0131664513de4666ef1f33dd23df1
Jul 01 19:24:21.872223 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.872197    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh"]
Jul 01 19:24:21.929157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:21.929134    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dcb499546-hckkd"]
Jul 01 19:24:22.148002 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:22.147963    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748" event={"ID":"c42e0814-fe61-43ab-a277-0804604a29de","Type":"ContainerStarted","Data":"ba254c9f031444998258d4e7dafbd40b08f265bba504a97bd51c5092f03a1d6f"}
Jul 01 19:24:22.149235 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:22.149211    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4" event={"ID":"74d7549c-4bc1-4749-ba0d-ac34e7d9457a","Type":"ContainerStarted","Data":"a997068930e6b3ef5a96e097d3d494f43fe0131664513de4666ef1f33dd23df1"}
Jul 01 19:24:22.943638 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:22.943574    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53dde71e_6f80_4d73_b4b1_8f3395466d98.slice/crio-bf64256c4f17884e3e0c7b511f71cd2243cfa716fcf2cf9749df7fe6766f4a86 WatchSource:0}: Error finding container bf64256c4f17884e3e0c7b511f71cd2243cfa716fcf2cf9749df7fe6766f4a86: Status 404 returned error can't find the container with id bf64256c4f17884e3e0c7b511f71cd2243cfa716fcf2cf9749df7fe6766f4a86
Jul 01 19:24:22.944538 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:22.944517    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21836a16_868d_46fb_93e8_5da6f6b471f7.slice/crio-f30b09c4caa3079cfd42e2b6e9baa1b7bae254f661e279483aef598fd123092f WatchSource:0}: Error finding container f30b09c4caa3079cfd42e2b6e9baa1b7bae254f661e279483aef598fd123092f: Status 404 returned error can't find the container with id f30b09c4caa3079cfd42e2b6e9baa1b7bae254f661e279483aef598fd123092f
Jul 01 19:24:23.156387 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:23.156336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwgxh" event={"ID":"43fa70b6-35bc-49e9-83ba-44aed748412c","Type":"ContainerStarted","Data":"e8fcc37ac676a6584d18481471a3b4c16ca12fa009c6a3c4a53f47d948a64362"}
Jul 01 19:24:23.160709 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:23.160282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dcb499546-hckkd" event={"ID":"21836a16-868d-46fb-93e8-5da6f6b471f7","Type":"ContainerStarted","Data":"418f1b967849b90940f457fae209599fa90ac4df109ade52698df382a0f08392"}
Jul 01 19:24:23.160709 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:23.160351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dcb499546-hckkd" event={"ID":"21836a16-868d-46fb-93e8-5da6f6b471f7","Type":"ContainerStarted","Data":"f30b09c4caa3079cfd42e2b6e9baa1b7bae254f661e279483aef598fd123092f"}
Jul 01 19:24:23.160709 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:23.160600    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:23.163157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:23.163129    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh" event={"ID":"53dde71e-6f80-4d73-b4b1-8f3395466d98","Type":"ContainerStarted","Data":"bf64256c4f17884e3e0c7b511f71cd2243cfa716fcf2cf9749df7fe6766f4a86"}
Jul 01 19:24:23.214618 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:23.214504    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hwgxh" podStartSLOduration=17.309900058 podStartE2EDuration="25.214482495s" totalImagesPullingTime="7.904582437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:23.214153244 +0000 UTC m=+59.762999623" watchObservedRunningTime="2026-07-01 19:24:23.214482495 +0000 UTC m=+59.763328854"
Jul 01 19:24:25.059913 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:25.059879    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcgrj"
Jul 01 19:24:25.101737 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:25.101679    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7dcb499546-hckkd" podStartSLOduration=4.101654518 podStartE2EDuration="4.101654518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:23.275382097 +0000 UTC m=+59.824228465" watchObservedRunningTime="2026-07-01 19:24:25.101654518 +0000 UTC m=+61.650500873"
Jul 01 19:24:29.188264 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.188221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748" event={"ID":"c42e0814-fe61-43ab-a277-0804604a29de","Type":"ContainerStarted","Data":"4d552a55fa259b9eaf0d535352c310ebb32371518b00930f9faf847ca6074784"}
Jul 01 19:24:29.188747 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.188426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:29.189993 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.189956    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh" event={"ID":"53dde71e-6f80-4d73-b4b1-8f3395466d98","Type":"ContainerStarted","Data":"d8b337d413ab1df496e12be0f159dcaa331f64ac9b4c84e5185907ade6ff45f0"}
Jul 01 19:24:29.190482 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.190462    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748"
Jul 01 19:24:29.191907 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.191883    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4" event={"ID":"74d7549c-4bc1-4749-ba0d-ac34e7d9457a","Type":"ContainerStarted","Data":"17b6e8a912cb054acf15efeaebaeb3769d93fb6bd36192a8e4dedb63058aa6ad"}
Jul 01 19:24:29.191981 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.191922    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4" event={"ID":"74d7549c-4bc1-4749-ba0d-ac34e7d9457a","Type":"ContainerStarted","Data":"c887147e5ed60478e23cfacf83d9daf28528a112cab37cba5e97ddf450f10392"}
Jul 01 19:24:29.191981 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.191942    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4" event={"ID":"74d7549c-4bc1-4749-ba0d-ac34e7d9457a","Type":"ContainerStarted","Data":"c1bb39cce42f6d58efd87a79adc68298c47f6cb6b6a46eed917164eceec65391"}
Jul 01 19:24:29.204484 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.204442    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bf669d4c8-c6748" podStartSLOduration=1.720226835 podStartE2EDuration="8.204429573s" totalImagesPullingTime="6.484202738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:29.202953084 +0000 UTC m=+65.751799459" watchObservedRunningTime="2026-07-01 19:24:29.204429573 +0000 UTC m=+65.753275948"
Jul 01 19:24:29.217356 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.217321    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746c5446db-h5rqh" podStartSLOduration=2.852004975 podStartE2EDuration="8.217310147s" totalImagesPullingTime="5.365305172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:29.216643065 +0000 UTC m=+65.765489439" watchObservedRunningTime="2026-07-01 19:24:29.217310147 +0000 UTC m=+65.766156522"
Jul 01 19:24:29.234121 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.234086    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-766fbbd9d-57hs4" podStartSLOduration=1.761586288 podStartE2EDuration="8.234073854s" totalImagesPullingTime="6.472487566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:29.233325808 +0000 UTC m=+65.782172182" watchObservedRunningTime="2026-07-01 19:24:29.234073854 +0000 UTC m=+65.782920229"
Jul 01 19:24:29.680426 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.680359    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:24:29.680612 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.680514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:24:29.682916 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.682895    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 19:24:29.682990 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.682924    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 19:24:29.693234 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.693207    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed27bed-2de0-4745-8ff1-b0c45f6a90ef-metrics-certs\") pod \"network-metrics-daemon-qw5hn\" (UID: \"aed27bed-2de0-4745-8ff1-b0c45f6a90ef\") " pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:24:29.693385 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.693202    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d-original-pull-secret\") pod \"global-pull-secret-syncer-4wtck\" (UID: \"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d\") " pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:24:29.824614 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.824568    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wtck"
Jul 01 19:24:29.840248 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.840216    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4nv7g\""
Jul 01 19:24:29.848520 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.848491    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5hn"
Jul 01 19:24:29.882680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.882649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:24:29.885480 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.885439    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 19:24:29.895480 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.895457    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 19:24:29.907121 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.906931    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxfz\" (UniqueName: \"kubernetes.io/projected/017149d2-7907-4bb3-8a16-780616d86470-kube-api-access-2jxfz\") pod \"network-check-target-bdsqr\" (UID: \"017149d2-7907-4bb3-8a16-780616d86470\") " pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:24:29.961051 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.961017    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4wtck"]
Jul 01 19:24:29.964471 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:29.964444    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac905aae_0eb4_4ad7_9d9c_70da0d4d1b2d.slice/crio-015a037476f71349f4dde36de4424c46bf6fe9b1375a991ae9af7d926d2301fe WatchSource:0}: Error finding container 015a037476f71349f4dde36de4424c46bf6fe9b1375a991ae9af7d926d2301fe: Status 404 returned error can't find the container with id 015a037476f71349f4dde36de4424c46bf6fe9b1375a991ae9af7d926d2301fe
Jul 01 19:24:29.987981 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:29.987956    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qw5hn"]
Jul 01 19:24:29.990951 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:29.990928    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed27bed_2de0_4745_8ff1_b0c45f6a90ef.slice/crio-925f12ac7ad48de13648aacef383c126be21d7cafc785821addfbb6d52b8c9c1 WatchSource:0}: Error finding container 925f12ac7ad48de13648aacef383c126be21d7cafc785821addfbb6d52b8c9c1: Status 404 returned error can't find the container with id 925f12ac7ad48de13648aacef383c126be21d7cafc785821addfbb6d52b8c9c1
Jul 01 19:24:30.133776 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:30.133739    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-64bmf\""
Jul 01 19:24:30.141851 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:30.141814    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:24:30.144694 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:30.144673    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l7bpz"
Jul 01 19:24:30.197396 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:30.197304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4wtck" event={"ID":"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d","Type":"ContainerStarted","Data":"015a037476f71349f4dde36de4424c46bf6fe9b1375a991ae9af7d926d2301fe"}
Jul 01 19:24:30.199770 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:30.199725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qw5hn" event={"ID":"aed27bed-2de0-4745-8ff1-b0c45f6a90ef","Type":"ContainerStarted","Data":"925f12ac7ad48de13648aacef383c126be21d7cafc785821addfbb6d52b8c9c1"}
Jul 01 19:24:30.287835 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:30.287802    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bdsqr"]
Jul 01 19:24:30.290614 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:30.290590    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017149d2_7907_4bb3_8a16_780616d86470.slice/crio-f49ab456716d9bc5652af8bc084e2828795d389d3cd5b41d36431d0825f343ae WatchSource:0}: Error finding container f49ab456716d9bc5652af8bc084e2828795d389d3cd5b41d36431d0825f343ae: Status 404 returned error can't find the container with id f49ab456716d9bc5652af8bc084e2828795d389d3cd5b41d36431d0825f343ae
Jul 01 19:24:31.205130 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:31.205054    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bdsqr" event={"ID":"017149d2-7907-4bb3-8a16-780616d86470","Type":"ContainerStarted","Data":"f49ab456716d9bc5652af8bc084e2828795d389d3cd5b41d36431d0825f343ae"}
Jul 01 19:24:31.385013 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:31.384956    2566 patch_prober.go:28] interesting pod/image-registry-6f88cc5f4-684dq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 01 19:24:31.385187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:31.385037    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" podUID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:24:32.211329 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:32.211292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qw5hn" event={"ID":"aed27bed-2de0-4745-8ff1-b0c45f6a90ef","Type":"ContainerStarted","Data":"33f790fac047ab6eccbc0a6cad14d34d4e933ed9888bdd0077b4be15c0c82b01"}
Jul 01 19:24:32.211801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:32.211358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qw5hn" event={"ID":"aed27bed-2de0-4745-8ff1-b0c45f6a90ef","Type":"ContainerStarted","Data":"8a68ca563109a704af9c97c93f077246a1c346c85c2cdde736e38ca05a3721fe"}
Jul 01 19:24:32.243386 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:32.243317    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qw5hn" podStartSLOduration=66.78723311 podStartE2EDuration="1m8.243293781s" totalImagesPullingTime="1.456060671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:32.241732633 +0000 UTC m=+68.790579008" watchObservedRunningTime="2026-07-01 19:24:32.243293781 +0000 UTC m=+68.792140157"
Jul 01 19:24:35.227755 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:35.227722    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4wtck" event={"ID":"ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d","Type":"ContainerStarted","Data":"287ccf94050a25cc6fdb84eaddd193489019a1ca0d4cd2bb3788eab85c5bddfe"}
Jul 01 19:24:35.229213 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:35.229188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bdsqr" event={"ID":"017149d2-7907-4bb3-8a16-780616d86470","Type":"ContainerStarted","Data":"20e5b7f4c0f8cc5b1bafe6263e8e02142436fa8da73572b10f9340578919f55a"}
Jul 01 19:24:35.229318 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:35.229240    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:24:35.265849 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:35.265766    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4wtck" podStartSLOduration=66.246401626 podStartE2EDuration="1m11.265753309s" totalImagesPullingTime="5.019351683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:35.264158813 +0000 UTC m=+71.813005194" watchObservedRunningTime="2026-07-01 19:24:35.265753309 +0000 UTC m=+71.814599683"
Jul 01 19:24:35.285323 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:35.285276    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bdsqr" podStartSLOduration=66.595268983 podStartE2EDuration="1m11.285259227s" totalImagesPullingTime="4.689990244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:35.284023898 +0000 UTC m=+71.832870273" watchObservedRunningTime="2026-07-01 19:24:35.285259227 +0000 UTC m=+71.834105603"
Jul 01 19:24:37.617571 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.617538    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pjdmz"]
Jul 01 19:24:37.619867 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.619843    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.622650 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.622622    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 19:24:37.622888 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.622868    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bfvnn\""
Jul 01 19:24:37.623207 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.623190    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 19:24:37.623492 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.623476    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 19:24:37.623601 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.623579    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 19:24:37.623769 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.623752    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 19:24:37.624507 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.624488    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 19:24:37.748477 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748434    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-sys\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748548    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27xh\" (UniqueName: \"kubernetes.io/projected/be1b3505-bca3-4fda-8e84-7c883319a4c1-kube-api-access-w27xh\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be1b3505-bca3-4fda-8e84-7c883319a4c1-metrics-client-ca\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748645    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-textfile\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-tls\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748889 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-wtmp\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748889 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748763    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.748889 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.748810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-root\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.849916 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.849881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-sys\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.849920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27xh\" (UniqueName: \"kubernetes.io/projected/be1b3505-bca3-4fda-8e84-7c883319a4c1-kube-api-access-w27xh\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.849986    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-sys\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850028    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be1b3505-bca3-4fda-8e84-7c883319a4c1-metrics-client-ca\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-textfile\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-tls\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850337 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-wtmp\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850337 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850337 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-root\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850337 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850502 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850333    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-wtmp\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850502 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850337    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be1b3505-bca3-4fda-8e84-7c883319a4c1-root\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850502 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:37.850443    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 01 19:24:37.850502 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850468    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-textfile\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850693 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:37.850522    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-tls podName:be1b3505-bca3-4fda-8e84-7c883319a4c1 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:38.350500638 +0000 UTC m=+74.899346998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-tls") pod "node-exporter-pjdmz" (UID: "be1b3505-bca3-4fda-8e84-7c883319a4c1") : secret "node-exporter-tls" not found
Jul 01 19:24:37.850878 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850856    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be1b3505-bca3-4fda-8e84-7c883319a4c1-metrics-client-ca\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.850992 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.850881    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.852828 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.852806    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:37.881912 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:37.881850    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27xh\" (UniqueName: \"kubernetes.io/projected/be1b3505-bca3-4fda-8e84-7c883319a4c1-kube-api-access-w27xh\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:38.355755 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:38.355718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-tls\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:38.358391 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:38.358354    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be1b3505-bca3-4fda-8e84-7c883319a4c1-node-exporter-tls\") pod \"node-exporter-pjdmz\" (UID: \"be1b3505-bca3-4fda-8e84-7c883319a4c1\") " pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:38.529588 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:38.529555    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pjdmz"
Jul 01 19:24:38.539688 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:24:38.539654    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1b3505_bca3_4fda_8e84_7c883319a4c1.slice/crio-b2ddae4070b55e79177db333329dcbb70b1a19e28db61f3777a91970b4ccc650 WatchSource:0}: Error finding container b2ddae4070b55e79177db333329dcbb70b1a19e28db61f3777a91970b4ccc650: Status 404 returned error can't find the container with id b2ddae4070b55e79177db333329dcbb70b1a19e28db61f3777a91970b4ccc650
Jul 01 19:24:39.245706 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:39.245659    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pjdmz" event={"ID":"be1b3505-bca3-4fda-8e84-7c883319a4c1","Type":"ContainerStarted","Data":"b2ddae4070b55e79177db333329dcbb70b1a19e28db61f3777a91970b4ccc650"}
Jul 01 19:24:40.250435 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:40.250393    2566 generic.go:356] "Generic (PLEG): container finished" podID="be1b3505-bca3-4fda-8e84-7c883319a4c1" containerID="9000ef23653803e0010cda9516e12870befeee160c51ef60026c08c80b569cc9" exitCode=0
Jul 01 19:24:40.250893 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:40.250464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pjdmz" event={"ID":"be1b3505-bca3-4fda-8e84-7c883319a4c1","Type":"ContainerDied","Data":"9000ef23653803e0010cda9516e12870befeee160c51ef60026c08c80b569cc9"}
Jul 01 19:24:41.255833 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:41.255792    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pjdmz" event={"ID":"be1b3505-bca3-4fda-8e84-7c883319a4c1","Type":"ContainerStarted","Data":"802609c8e29a607cff4f05ff0a5ea1dd6170a40693e3d93ce70f5a48b0f96c16"}
Jul 01 19:24:41.256226 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:41.255859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pjdmz" event={"ID":"be1b3505-bca3-4fda-8e84-7c883319a4c1","Type":"ContainerStarted","Data":"dfc072469748d0e16890a7ca3074f2fdf06af179f116ec3853ce2a61ad432c2c"}
Jul 01 19:24:41.276119 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:41.276075    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pjdmz" podStartSLOduration=3.199989205 podStartE2EDuration="4.27605928s" totalImagesPullingTime="1.004895768s" totalInitContainerRuntime="71.174307ms" isStatefulPod=true podCreationTimestamp="2026-07-01 19:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:41.275572406 +0000 UTC m=+77.824418782" watchObservedRunningTime="2026-07-01 19:24:41.27605928 +0000 UTC m=+77.824905654"
Jul 01 19:24:41.382825 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:41.382799    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:44.174038 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:44.173924    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7dcb499546-hckkd"
Jul 01 19:24:46.406664 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.406625    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" podUID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" containerName="registry" containerID="cri-o://85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e" gracePeriod=30
Jul 01 19:24:46.644768 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.644745    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:46.725149 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725074    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-certificates\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725149 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725110    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725149 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725142    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djhng\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-kube-api-access-djhng\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725405 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725163    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-image-registry-private-configuration\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725405 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725204    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92c4f69c-06a5-46aa-abd0-0eca63ca7440-ca-trust-extracted\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725405 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725228    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-installation-pull-secrets\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725405 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725254    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-trusted-ca\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725611 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725421    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-bound-sa-token\") pod \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\" (UID: \"92c4f69c-06a5-46aa-abd0-0eca63ca7440\") "
Jul 01 19:24:46.725611 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725601    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:24:46.725728 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725692    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-certificates\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.725992 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.725939    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:24:46.727790 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.727757    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:24:46.727906 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.727861    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:24:46.728160 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.728122    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:24:46.728259 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.728224    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:24:46.728403 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.728361    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-kube-api-access-djhng" (OuterVolumeSpecName: "kube-api-access-djhng") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "kube-api-access-djhng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:24:46.737120 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.737094    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92c4f69c-06a5-46aa-abd0-0eca63ca7440-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "92c4f69c-06a5-46aa-abd0-0eca63ca7440" (UID: "92c4f69c-06a5-46aa-abd0-0eca63ca7440"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:24:46.826731 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826677    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-registry-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.826731 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826724    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djhng\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-kube-api-access-djhng\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.826731 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826741    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-image-registry-private-configuration\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.826969 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826755    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92c4f69c-06a5-46aa-abd0-0eca63ca7440-ca-trust-extracted\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.826969 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826766    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92c4f69c-06a5-46aa-abd0-0eca63ca7440-installation-pull-secrets\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.826969 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826778    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92c4f69c-06a5-46aa-abd0-0eca63ca7440-trusted-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:46.826969 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:46.826788    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92c4f69c-06a5-46aa-abd0-0eca63ca7440-bound-sa-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:24:47.277642 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.277607    2566 generic.go:356] "Generic (PLEG): container finished" podID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" containerID="85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e" exitCode=0
Jul 01 19:24:47.277822 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.277655    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq"
Jul 01 19:24:47.277822 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.277681    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" event={"ID":"92c4f69c-06a5-46aa-abd0-0eca63ca7440","Type":"ContainerDied","Data":"85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e"}
Jul 01 19:24:47.277822 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.277732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f88cc5f4-684dq" event={"ID":"92c4f69c-06a5-46aa-abd0-0eca63ca7440","Type":"ContainerDied","Data":"71c362620c82ccd2e3985a815c2c105d706a9668a4a798b60f78f118e1d63c05"}
Jul 01 19:24:47.277822 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.277757    2566 scope.go:117] "RemoveContainer" containerID="85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e"
Jul 01 19:24:47.286422 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.286401    2566 scope.go:117] "RemoveContainer" containerID="85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e"
Jul 01 19:24:47.286698 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:24:47.286669    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e\": container with ID starting with 85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e not found: ID does not exist" containerID="85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e"
Jul 01 19:24:47.286752 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.286707    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e"} err="failed to get container status \"85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e\": rpc error: code = NotFound desc = could not find container \"85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e\": container with ID starting with 85e36281e598dff9b1b4acee4e5737575f76954624377b18717cfdaa7376265e not found: ID does not exist"
Jul 01 19:24:47.298052 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.298027    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f88cc5f4-684dq"]
Jul 01 19:24:47.305950 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.305924    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f88cc5f4-684dq"]
Jul 01 19:24:47.917609 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:24:47.917573    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" path="/var/lib/kubelet/pods/92c4f69c-06a5-46aa-abd0-0eca63ca7440/volumes"
Jul 01 19:25:03.181025 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:25:03.180948    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-646vp_3bf8d620-9462-42bb-a12b-8204bd79abab/serve-healthcheck-canary/0.log"
Jul 01 19:25:06.235879 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:25:06.235843    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bdsqr"
Jul 01 19:28:23.852062 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:28:23.852031    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:28:23.854791 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:28:23.854756    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 19:28:23.856207 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:28:23.856187    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:29:42.743072 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.742977    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-tfnfs"]
Jul 01 19:29:42.743651 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.743207    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" containerName="registry"
Jul 01 19:29:42.743651 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.743219    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" containerName="registry"
Jul 01 19:29:42.743651 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.743264    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c4f69c-06a5-46aa-abd0-0eca63ca7440" containerName="registry"
Jul 01 19:29:42.745086 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.745069    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:42.749211 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.749183    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 01 19:29:42.749387 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.749353    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 01 19:29:42.750173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.750154    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-6pb2g\""
Jul 01 19:29:42.750278 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.750154    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 01 19:29:42.760949 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.760929    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-tfnfs"]
Jul 01 19:29:42.818223 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.818187    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967fe994-d440-44d4-bfb0-cac168643fc2-tls-certs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:42.818430 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.818230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr2qs\" (UniqueName: \"kubernetes.io/projected/967fe994-d440-44d4-bfb0-cac168643fc2-kube-api-access-fr2qs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:42.919599 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.919554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967fe994-d440-44d4-bfb0-cac168643fc2-tls-certs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:42.919787 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.919614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr2qs\" (UniqueName: \"kubernetes.io/projected/967fe994-d440-44d4-bfb0-cac168643fc2-kube-api-access-fr2qs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:42.919787 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:29:42.919698    2566 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 01 19:29:42.919787 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:29:42.919766    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967fe994-d440-44d4-bfb0-cac168643fc2-tls-certs podName:967fe994-d440-44d4-bfb0-cac168643fc2 nodeName:}" failed. No retries permitted until 2026-07-01 19:29:43.419750176 +0000 UTC m=+379.968596533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/967fe994-d440-44d4-bfb0-cac168643fc2-tls-certs") pod "model-serving-api-fd65d7d6-tfnfs" (UID: "967fe994-d440-44d4-bfb0-cac168643fc2") : secret "model-serving-api-tls" not found
Jul 01 19:29:42.933835 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:42.933798    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr2qs\" (UniqueName: \"kubernetes.io/projected/967fe994-d440-44d4-bfb0-cac168643fc2-kube-api-access-fr2qs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:43.423805 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:43.423767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967fe994-d440-44d4-bfb0-cac168643fc2-tls-certs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:43.426311 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:43.426279    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967fe994-d440-44d4-bfb0-cac168643fc2-tls-certs\") pod \"model-serving-api-fd65d7d6-tfnfs\" (UID: \"967fe994-d440-44d4-bfb0-cac168643fc2\") " pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:43.654767 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:43.654727    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:43.812090 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:43.812034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-tfnfs"]
Jul 01 19:29:43.814662 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:29:43.814631    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967fe994_d440_44d4_bfb0_cac168643fc2.slice/crio-948444e8acfb4043efabd1672b50437be9d141bf1bb04cad106ed57f05ef7b19 WatchSource:0}: Error finding container 948444e8acfb4043efabd1672b50437be9d141bf1bb04cad106ed57f05ef7b19: Status 404 returned error can't find the container with id 948444e8acfb4043efabd1672b50437be9d141bf1bb04cad106ed57f05ef7b19
Jul 01 19:29:43.816565 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:43.816546    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:29:44.249518 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:44.249474    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-tfnfs" event={"ID":"967fe994-d440-44d4-bfb0-cac168643fc2","Type":"ContainerStarted","Data":"948444e8acfb4043efabd1672b50437be9d141bf1bb04cad106ed57f05ef7b19"}
Jul 01 19:29:46.257536 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:46.257450    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-tfnfs" event={"ID":"967fe994-d440-44d4-bfb0-cac168643fc2","Type":"ContainerStarted","Data":"c9af5c4d365a64181e965dff3d1b5411770bb26f19b2e273322baedacae04799"}
Jul 01 19:29:46.257888 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:46.257586    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:29:46.277913 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:46.277865    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-tfnfs" podStartSLOduration=2.191747073 podStartE2EDuration="4.27785071s" totalImagesPullingTime="2.086103637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:29:46.276438362 +0000 UTC m=+382.825284738" watchObservedRunningTime="2026-07-01 19:29:46.27785071 +0000 UTC m=+382.826697135"
Jul 01 19:29:57.264569 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:29:57.264539    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-tfnfs"
Jul 01 19:31:30.942094 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.942060    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"]
Jul 01 19:31:30.945056 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.945039    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:30.949157 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.949137    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 19:31:30.949441 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.949426    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 19:31:30.949749 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.949732    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8fd9c-predictor-serving-cert\""
Jul 01 19:31:30.950499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.950481    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tw4nc\""
Jul 01 19:31:30.953927 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.953912    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8fd9c-kube-rbac-proxy-sar-config\""
Jul 01 19:31:30.969669 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:30.969643    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"]
Jul 01 19:31:31.005858 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.005829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/90646572-b409-4a64-b093-551a90805d07-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.006005 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.005876    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fmt4\" (UniqueName: \"kubernetes.io/projected/90646572-b409-4a64-b093-551a90805d07-kube-api-access-4fmt4\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.006068 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.006053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.106563 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.106533    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.106753 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.106581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/90646572-b409-4a64-b093-551a90805d07-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.106753 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.106612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fmt4\" (UniqueName: \"kubernetes.io/projected/90646572-b409-4a64-b093-551a90805d07-kube-api-access-4fmt4\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.106753 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:31:31.106701    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-serving-cert: secret "success-200-isvc-8fd9c-predictor-serving-cert" not found
Jul 01 19:31:31.106923 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:31:31.106809    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls podName:90646572-b409-4a64-b093-551a90805d07 nodeName:}" failed. No retries permitted until 2026-07-01 19:31:31.606787178 +0000 UTC m=+488.155633534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls") pod "success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" (UID: "90646572-b409-4a64-b093-551a90805d07") : secret "success-200-isvc-8fd9c-predictor-serving-cert" not found
Jul 01 19:31:31.107531 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.107500    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/90646572-b409-4a64-b093-551a90805d07-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.129762 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.129731    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fmt4\" (UniqueName: \"kubernetes.io/projected/90646572-b409-4a64-b093-551a90805d07-kube-api-access-4fmt4\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.610209 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.610171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.612770 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.612751    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls\") pod \"success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") " pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:31.855879 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:31.855825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:32.019791 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:32.019739    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"]
Jul 01 19:31:32.023269 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:31:32.023237    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90646572_b409_4a64_b093_551a90805d07.slice/crio-7546733aea0de68cbbaffd4a0ae7f3c0a862b7aebffbb8c4127d4bec4dd2f7d1 WatchSource:0}: Error finding container 7546733aea0de68cbbaffd4a0ae7f3c0a862b7aebffbb8c4127d4bec4dd2f7d1: Status 404 returned error can't find the container with id 7546733aea0de68cbbaffd4a0ae7f3c0a862b7aebffbb8c4127d4bec4dd2f7d1
Jul 01 19:31:32.608023 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:32.607987    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" event={"ID":"90646572-b409-4a64-b093-551a90805d07","Type":"ContainerStarted","Data":"7546733aea0de68cbbaffd4a0ae7f3c0a862b7aebffbb8c4127d4bec4dd2f7d1"}
Jul 01 19:31:44.662016 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:44.661929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" event={"ID":"90646572-b409-4a64-b093-551a90805d07","Type":"ContainerStarted","Data":"b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8"}
Jul 01 19:31:47.673960 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:47.673919    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" event={"ID":"90646572-b409-4a64-b093-551a90805d07","Type":"ContainerStarted","Data":"79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2"}
Jul 01 19:31:47.674355 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:47.674047    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:47.716315 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:47.716258    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podStartSLOduration=3.179819152 podStartE2EDuration="17.716243176s" totalImagesPullingTime="14.536424024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:31:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:31:47.713952342 +0000 UTC m=+504.262798718" watchObservedRunningTime="2026-07-01 19:31:47.716243176 +0000 UTC m=+504.265089550"
Jul 01 19:31:48.677749 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:48.677705    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:48.679069 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:48.679046    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 01 19:31:49.680647 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:49.680601    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 01 19:31:54.686526 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:54.686491    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:31:54.687026 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:31:54.686999    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 01 19:32:04.687804 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:32:04.687765    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 01 19:32:14.687802 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:32:14.687715    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 01 19:32:24.687237 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:32:24.687195    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 01 19:32:34.687579 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:32:34.687548    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:33:01.269239 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.269204    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"]
Jul 01 19:33:01.269807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.269463    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container" containerID="cri-o://b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8" gracePeriod=30
Jul 01 19:33:01.269807 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.269553    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kube-rbac-proxy" containerID="cri-o://79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2" gracePeriod=30
Jul 01 19:33:01.307970 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.307943    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"]
Jul 01 19:33:01.310104 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.310084    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.312834 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.312811    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-834d0-predictor-serving-cert\""
Jul 01 19:33:01.312928 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.312822    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-834d0-kube-rbac-proxy-sar-config\""
Jul 01 19:33:01.326004 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.325978    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"]
Jul 01 19:33:01.381969 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.381933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bss95\" (UniqueName: \"kubernetes.io/projected/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-api-access-bss95\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.382116 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.382041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.382116 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.382074    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.483337 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.483300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.483542 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.483347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bss95\" (UniqueName: \"kubernetes.io/projected/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-api-access-bss95\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.483542 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.483409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.483542 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:33:01.483517    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-834d0-predictor-serving-cert: secret "success-200-isvc-834d0-predictor-serving-cert" not found
Jul 01 19:33:01.483699 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:33:01.483594    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls podName:52822648-3dee-484a-bc6f-6d982fcfa87f nodeName:}" failed. No retries permitted until 2026-07-01 19:33:01.98357479 +0000 UTC m=+578.532421153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls") pod "success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" (UID: "52822648-3dee-484a-bc6f-6d982fcfa87f") : secret "success-200-isvc-834d0-predictor-serving-cert" not found
Jul 01 19:33:01.484108 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.484083    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.501069 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.501043    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bss95\" (UniqueName: \"kubernetes.io/projected/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-api-access-bss95\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.938926 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.938893    2566 generic.go:356] "Generic (PLEG): container finished" podID="90646572-b409-4a64-b093-551a90805d07" containerID="79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2" exitCode=2
Jul 01 19:33:01.939093 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.938949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" event={"ID":"90646572-b409-4a64-b093-551a90805d07","Type":"ContainerDied","Data":"79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2"}
Jul 01 19:33:01.986285 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.986252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:01.988750 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:01.988722    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls\") pod \"success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") " pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:02.219740 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.219642    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:02.353392 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.353341    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"]
Jul 01 19:33:02.355966 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:33:02.355928    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52822648_3dee_484a_bc6f_6d982fcfa87f.slice/crio-af86d54cb58234e800e067433ebc330a1627906c6751d7b61c6602690e105737 WatchSource:0}: Error finding container af86d54cb58234e800e067433ebc330a1627906c6751d7b61c6602690e105737: Status 404 returned error can't find the container with id af86d54cb58234e800e067433ebc330a1627906c6751d7b61c6602690e105737
Jul 01 19:33:02.943990 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.943951    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" event={"ID":"52822648-3dee-484a-bc6f-6d982fcfa87f","Type":"ContainerStarted","Data":"bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5"}
Jul 01 19:33:02.944145 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.944023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" event={"ID":"52822648-3dee-484a-bc6f-6d982fcfa87f","Type":"ContainerStarted","Data":"3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46"}
Jul 01 19:33:02.944145 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.944058    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:02.944145 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.944085    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" event={"ID":"52822648-3dee-484a-bc6f-6d982fcfa87f","Type":"ContainerStarted","Data":"af86d54cb58234e800e067433ebc330a1627906c6751d7b61c6602690e105737"}
Jul 01 19:33:02.944145 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.944113    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:02.945499 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.945468    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:33:02.964105 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:02.964050    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podStartSLOduration=1.964034132 podStartE2EDuration="1.964034132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:33:02.962304502 +0000 UTC m=+579.511150877" watchObservedRunningTime="2026-07-01 19:33:02.964034132 +0000 UTC m=+579.512880507"
Jul 01 19:33:03.948291 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:03.948254    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:33:04.511034 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.511012    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:33:04.608882 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.608795    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/90646572-b409-4a64-b093-551a90805d07-kube-rbac-proxy-sar-config\") pod \"90646572-b409-4a64-b093-551a90805d07\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") "
Jul 01 19:33:04.608882 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.608865    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fmt4\" (UniqueName: \"kubernetes.io/projected/90646572-b409-4a64-b093-551a90805d07-kube-api-access-4fmt4\") pod \"90646572-b409-4a64-b093-551a90805d07\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") "
Jul 01 19:33:04.609085 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.608891    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls\") pod \"90646572-b409-4a64-b093-551a90805d07\" (UID: \"90646572-b409-4a64-b093-551a90805d07\") "
Jul 01 19:33:04.609193 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.609151    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90646572-b409-4a64-b093-551a90805d07-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "90646572-b409-4a64-b093-551a90805d07" (UID: "90646572-b409-4a64-b093-551a90805d07"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:33:04.611276 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.611256    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90646572-b409-4a64-b093-551a90805d07-kube-api-access-4fmt4" (OuterVolumeSpecName: "kube-api-access-4fmt4") pod "90646572-b409-4a64-b093-551a90805d07" (UID: "90646572-b409-4a64-b093-551a90805d07"). InnerVolumeSpecName "kube-api-access-4fmt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:33:04.611344 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.611321    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "90646572-b409-4a64-b093-551a90805d07" (UID: "90646572-b409-4a64-b093-551a90805d07"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:33:04.710028 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.709991    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fmt4\" (UniqueName: \"kubernetes.io/projected/90646572-b409-4a64-b093-551a90805d07-kube-api-access-4fmt4\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.710028 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.710020    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/90646572-b409-4a64-b093-551a90805d07-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.710028 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.710030    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/90646572-b409-4a64-b093-551a90805d07-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.953742 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.953639    2566 generic.go:356] "Generic (PLEG): container finished" podID="90646572-b409-4a64-b093-551a90805d07" containerID="b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8" exitCode=0
Jul 01 19:33:04.953742 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.953690    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"
Jul 01 19:33:04.953742 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.953700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" event={"ID":"90646572-b409-4a64-b093-551a90805d07","Type":"ContainerDied","Data":"b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8"}
Jul 01 19:33:04.954278 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.953763    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv" event={"ID":"90646572-b409-4a64-b093-551a90805d07","Type":"ContainerDied","Data":"7546733aea0de68cbbaffd4a0ae7f3c0a862b7aebffbb8c4127d4bec4dd2f7d1"}
Jul 01 19:33:04.954278 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.953797    2566 scope.go:117] "RemoveContainer" containerID="79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2"
Jul 01 19:33:04.962379 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.962346    2566 scope.go:117] "RemoveContainer" containerID="b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8"
Jul 01 19:33:04.969625 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.969605    2566 scope.go:117] "RemoveContainer" containerID="79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2"
Jul 01 19:33:04.969902 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:33:04.969881    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2\": container with ID starting with 79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2 not found: ID does not exist" containerID="79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2"
Jul 01 19:33:04.969973 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.969916    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2"} err="failed to get container status \"79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2\": rpc error: code = NotFound desc = could not find container \"79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2\": container with ID starting with 79f22d41a00c2f93fe8d59f94dfcbfaa203c953fc081c168210511be115e15c2 not found: ID does not exist"
Jul 01 19:33:04.969973 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.969942    2566 scope.go:117] "RemoveContainer" containerID="b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8"
Jul 01 19:33:04.970200 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:33:04.970182    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8\": container with ID starting with b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8 not found: ID does not exist" containerID="b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8"
Jul 01 19:33:04.970247 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.970208    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8"} err="failed to get container status \"b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8\": rpc error: code = NotFound desc = could not find container \"b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8\": container with ID starting with b6d7e88b8f54380f9d35265cfd85b2a66b825cbeda3398c20fe6fc668515bee8 not found: ID does not exist"
Jul 01 19:33:04.991247 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.991220    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"]
Jul 01 19:33:04.992974 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:04.992952    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fd9c-predictor-6c6b4fc89c-sbkzv"]
Jul 01 19:33:05.917564 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:05.917531    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90646572-b409-4a64-b093-551a90805d07" path="/var/lib/kubelet/pods/90646572-b409-4a64-b093-551a90805d07/volumes"
Jul 01 19:33:08.953553 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:08.953524    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:33:08.954303 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:08.954268    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:33:18.954840 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:18.954784    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:33:23.878107 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:23.878074    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:33:23.879936 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:23.879915    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:33:28.954990 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:28.954948    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:33:38.954416 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:38.954349    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:33:48.955141 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:33:48.955064    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:34:01.103041 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103005    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"]
Jul 01 19:34:01.103490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103337    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container"
Jul 01 19:34:01.103490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103351    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container"
Jul 01 19:34:01.103490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103399    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kube-rbac-proxy"
Jul 01 19:34:01.103490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103406    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kube-rbac-proxy"
Jul 01 19:34:01.103490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103463    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kserve-container"
Jul 01 19:34:01.103490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.103472    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="90646572-b409-4a64-b093-551a90805d07" containerName="kube-rbac-proxy"
Jul 01 19:34:01.105288 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.105271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.108631 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.108612    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-01ba2-kube-rbac-proxy-sar-config\""
Jul 01 19:34:01.108754 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.108608    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-01ba2-predictor-serving-cert\""
Jul 01 19:34:01.119847 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.119821    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"]
Jul 01 19:34:01.218266 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.218224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.218453 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.218279    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.218453 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.218344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn4mz\" (UniqueName: \"kubernetes.io/projected/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-api-access-cn4mz\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.319519 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.319481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.319670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.319567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.319670 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.319615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn4mz\" (UniqueName: \"kubernetes.io/projected/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-api-access-cn4mz\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.319670 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:34:01.319643    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-serving-cert: secret "success-200-isvc-01ba2-predictor-serving-cert" not found
Jul 01 19:34:01.319802 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:34:01.319724    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls podName:4fd4a47d-0c8b-4867-b55f-3a1857a88604 nodeName:}" failed. No retries permitted until 2026-07-01 19:34:01.819702392 +0000 UTC m=+638.368548765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls") pod "success-200-isvc-01ba2-predictor-547f47f477-gx5hl" (UID: "4fd4a47d-0c8b-4867-b55f-3a1857a88604") : secret "success-200-isvc-01ba2-predictor-serving-cert" not found
Jul 01 19:34:01.320547 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.320524    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.334108 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.334075    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn4mz\" (UniqueName: \"kubernetes.io/projected/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-api-access-cn4mz\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.823002 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.822963    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:01.825594 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:01.825566    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls\") pod \"success-200-isvc-01ba2-predictor-547f47f477-gx5hl\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") " pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:02.016377 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:02.016340    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:02.157450 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:02.155583    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"]
Jul 01 19:34:02.160411 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:34:02.160382    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd4a47d_0c8b_4867_b55f_3a1857a88604.slice/crio-81fd3e9881e1c50e3a973a24fe4a0344d1b0c72917394047620d95df1585a393 WatchSource:0}: Error finding container 81fd3e9881e1c50e3a973a24fe4a0344d1b0c72917394047620d95df1585a393: Status 404 returned error can't find the container with id 81fd3e9881e1c50e3a973a24fe4a0344d1b0c72917394047620d95df1585a393
Jul 01 19:34:03.160815 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.160721    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" event={"ID":"4fd4a47d-0c8b-4867-b55f-3a1857a88604","Type":"ContainerStarted","Data":"3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba"}
Jul 01 19:34:03.160815 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.160787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" event={"ID":"4fd4a47d-0c8b-4867-b55f-3a1857a88604","Type":"ContainerStarted","Data":"363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d"}
Jul 01 19:34:03.162280 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.160824    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:03.162280 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.160848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" event={"ID":"4fd4a47d-0c8b-4867-b55f-3a1857a88604","Type":"ContainerStarted","Data":"81fd3e9881e1c50e3a973a24fe4a0344d1b0c72917394047620d95df1585a393"}
Jul 01 19:34:03.162280 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.160878    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:03.162280 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.162120    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:34:03.179656 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:03.179600    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podStartSLOduration=2.179581104 podStartE2EDuration="2.179581104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:34:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:34:03.178473627 +0000 UTC m=+639.727320003" watchObservedRunningTime="2026-07-01 19:34:03.179581104 +0000 UTC m=+639.728427480"
Jul 01 19:34:04.163874 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:04.163834    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:34:09.168590 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:09.168560    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:34:09.169062 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:09.169036    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:34:19.169484 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:19.169439    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:34:29.169985 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:29.169947    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:34:39.170067 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:39.170018    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:34:49.170691 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:34:49.170663    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:38:23.898149 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:38:23.898081    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:38:23.898149 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:38:23.898081    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:42:16.238013 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.237978    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"]
Jul 01 19:42:16.249219 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.249185    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.251882 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.251775    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ac147-predictor-serving-cert\""
Jul 01 19:42:16.252068 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.251888    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ac147-kube-rbac-proxy-sar-config\""
Jul 01 19:42:16.255684 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.255660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"]
Jul 01 19:42:16.265034 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.265009    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"]
Jul 01 19:42:16.265303 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.265275    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" containerID="cri-o://3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46" gracePeriod=30
Jul 01 19:42:16.265429 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.265396    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kube-rbac-proxy" containerID="cri-o://bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5" gracePeriod=30
Jul 01 19:42:16.394274 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.394242    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-proxy-tls\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.394422 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.394290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v5kt\" (UniqueName: \"kubernetes.io/projected/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-api-access-8v5kt\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.394422 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.394321    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.494941 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.494852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.495102 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.494918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-proxy-tls\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.495102 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.494986    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v5kt\" (UniqueName: \"kubernetes.io/projected/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-api-access-8v5kt\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.495579 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.495541    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.497538 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.497515    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-proxy-tls\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.502887 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.502866    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v5kt\" (UniqueName: \"kubernetes.io/projected/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-api-access-8v5kt\") pod \"success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") " pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.562939 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.562897    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.709572 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.709542    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"]
Jul 01 19:42:16.712308 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:42:16.712281    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9acb56c_4c2e_48ff_9baf_86c9f9428d78.slice/crio-f59e814ccbde7595865013b29afdc35d0b68614c802a15e575a71ef4399e0d17 WatchSource:0}: Error finding container f59e814ccbde7595865013b29afdc35d0b68614c802a15e575a71ef4399e0d17: Status 404 returned error can't find the container with id f59e814ccbde7595865013b29afdc35d0b68614c802a15e575a71ef4399e0d17
Jul 01 19:42:16.884051 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.884015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" event={"ID":"e9acb56c-4c2e-48ff-9baf-86c9f9428d78","Type":"ContainerStarted","Data":"ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a"}
Jul 01 19:42:16.884222 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.884077    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" event={"ID":"e9acb56c-4c2e-48ff-9baf-86c9f9428d78","Type":"ContainerStarted","Data":"4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7"}
Jul 01 19:42:16.884222 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.884110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" event={"ID":"e9acb56c-4c2e-48ff-9baf-86c9f9428d78","Type":"ContainerStarted","Data":"f59e814ccbde7595865013b29afdc35d0b68614c802a15e575a71ef4399e0d17"}
Jul 01 19:42:16.884222 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.884146    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:16.885924 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.885897    2566 generic.go:356] "Generic (PLEG): container finished" podID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerID="bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5" exitCode=2
Jul 01 19:42:16.886029 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.885965    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" event={"ID":"52822648-3dee-484a-bc6f-6d982fcfa87f","Type":"ContainerDied","Data":"bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5"}
Jul 01 19:42:16.901766 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:16.901722    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podStartSLOduration=0.901709755 podStartE2EDuration="901.709755ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:42:16.900762969 +0000 UTC m=+1133.449609584" watchObservedRunningTime="2026-07-01 19:42:16.901709755 +0000 UTC m=+1133.450556129"
Jul 01 19:42:17.889965 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:17.889935    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:17.891282 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:17.891253    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 01 19:42:18.894000 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:18.893965    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 01 19:42:18.948949 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:18.948896    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.17:8643/healthz\": dial tcp 10.133.0.17:8643: connect: connection refused"
Jul 01 19:42:18.954386 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:18.954325    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 01 19:42:19.608240 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.608214    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:42:19.721616 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.721524    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bss95\" (UniqueName: \"kubernetes.io/projected/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-api-access-bss95\") pod \"52822648-3dee-484a-bc6f-6d982fcfa87f\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") "
Jul 01 19:42:19.721616 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.721605    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls\") pod \"52822648-3dee-484a-bc6f-6d982fcfa87f\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") "
Jul 01 19:42:19.721848 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.721666    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-rbac-proxy-sar-config\") pod \"52822648-3dee-484a-bc6f-6d982fcfa87f\" (UID: \"52822648-3dee-484a-bc6f-6d982fcfa87f\") "
Jul 01 19:42:19.722067 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.722037    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "52822648-3dee-484a-bc6f-6d982fcfa87f" (UID: "52822648-3dee-484a-bc6f-6d982fcfa87f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:42:19.723901 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.723870    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-api-access-bss95" (OuterVolumeSpecName: "kube-api-access-bss95") pod "52822648-3dee-484a-bc6f-6d982fcfa87f" (UID: "52822648-3dee-484a-bc6f-6d982fcfa87f"). InnerVolumeSpecName "kube-api-access-bss95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:42:19.723901 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.723882    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "52822648-3dee-484a-bc6f-6d982fcfa87f" (UID: "52822648-3dee-484a-bc6f-6d982fcfa87f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:42:19.822901 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.822848    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52822648-3dee-484a-bc6f-6d982fcfa87f-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:42:19.822901 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.822890    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:42:19.822901 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.822899    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bss95\" (UniqueName: \"kubernetes.io/projected/52822648-3dee-484a-bc6f-6d982fcfa87f-kube-api-access-bss95\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:42:19.898658 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.898615    2566 generic.go:356] "Generic (PLEG): container finished" podID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerID="3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46" exitCode=0
Jul 01 19:42:19.899116 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.898668    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" event={"ID":"52822648-3dee-484a-bc6f-6d982fcfa87f","Type":"ContainerDied","Data":"3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46"}
Jul 01 19:42:19.899116 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.898676    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"
Jul 01 19:42:19.899116 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.898726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l" event={"ID":"52822648-3dee-484a-bc6f-6d982fcfa87f","Type":"ContainerDied","Data":"af86d54cb58234e800e067433ebc330a1627906c6751d7b61c6602690e105737"}
Jul 01 19:42:19.899116 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.898757    2566 scope.go:117] "RemoveContainer" containerID="bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5"
Jul 01 19:42:19.907545 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.907527    2566 scope.go:117] "RemoveContainer" containerID="3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46"
Jul 01 19:42:19.914893 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.914875    2566 scope.go:117] "RemoveContainer" containerID="bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5"
Jul 01 19:42:19.915199 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:42:19.915179    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5\": container with ID starting with bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5 not found: ID does not exist" containerID="bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5"
Jul 01 19:42:19.915285 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.915206    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5"} err="failed to get container status \"bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5\": rpc error: code = NotFound desc = could not find container \"bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5\": container with ID starting with bcb190a88a39b683f861c308dec0598dbb43345c6206b4189b01a0de610fb9a5 not found: ID does not exist"
Jul 01 19:42:19.915285 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.915222    2566 scope.go:117] "RemoveContainer" containerID="3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46"
Jul 01 19:42:19.915535 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:42:19.915456    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46\": container with ID starting with 3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46 not found: ID does not exist" containerID="3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46"
Jul 01 19:42:19.915535 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.915475    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46"} err="failed to get container status \"3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46\": rpc error: code = NotFound desc = could not find container \"3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46\": container with ID starting with 3925bae5f6816f6de11c54001de214534688974dd978f82fe5dc579486123c46 not found: ID does not exist"
Jul 01 19:42:19.919890 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.919869    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"]
Jul 01 19:42:19.923618 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:19.923600    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-834d0-predictor-6bdb44c5f9-tdn8l"]
Jul 01 19:42:21.917644 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:21.917611    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" path="/var/lib/kubelet/pods/52822648-3dee-484a-bc6f-6d982fcfa87f/volumes"
Jul 01 19:42:23.898282 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:23.898254    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:42:23.898867 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:23.898835    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 01 19:42:33.899639 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:33.899599    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 01 19:42:43.899153 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:43.899113    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 01 19:42:53.899699 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:42:53.899660    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 01 19:43:03.899525 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:03.899491    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:43:15.983481 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:15.983447    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"]
Jul 01 19:43:15.983966 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:15.983708    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" containerID="cri-o://363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d" gracePeriod=30
Jul 01 19:43:15.983966 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:15.983734    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kube-rbac-proxy" containerID="cri-o://3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba" gracePeriod=30
Jul 01 19:43:16.001069 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001045    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"]
Jul 01 19:43:16.001307 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001295    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container"
Jul 01 19:43:16.001354 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001309    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container"
Jul 01 19:43:16.001354 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001328    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kube-rbac-proxy"
Jul 01 19:43:16.001354 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001333    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kube-rbac-proxy"
Jul 01 19:43:16.001469 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001395    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kserve-container"
Jul 01 19:43:16.001469 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.001404    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="52822648-3dee-484a-bc6f-6d982fcfa87f" containerName="kube-rbac-proxy"
Jul 01 19:43:16.004448 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.004426    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.006576 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.006557    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-46ddd-predictor-serving-cert\""
Jul 01 19:43:16.006663 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.006646    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-46ddd-kube-rbac-proxy-sar-config\""
Jul 01 19:43:16.015768 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.015748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"]
Jul 01 19:43:16.049110 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.049072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.049272 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.049133    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wpbm\" (UniqueName: \"kubernetes.io/projected/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-api-access-5wpbm\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.049272 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.049205    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.149659 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.149620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.149838 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.149671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wpbm\" (UniqueName: \"kubernetes.io/projected/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-api-access-5wpbm\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.149838 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.149694    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.149949 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:16.149902    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-serving-cert: secret "success-200-isvc-46ddd-predictor-serving-cert" not found
Jul 01 19:43:16.149998 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:16.149968    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls podName:110eb4e9-8a92-4762-9350-7be4e8d78602 nodeName:}" failed. No retries permitted until 2026-07-01 19:43:16.649952232 +0000 UTC m=+1193.198798586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls") pod "success-200-isvc-46ddd-predictor-6647b68fc-zrszv" (UID: "110eb4e9-8a92-4762-9350-7be4e8d78602") : secret "success-200-isvc-46ddd-predictor-serving-cert" not found
Jul 01 19:43:16.150410 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.150360    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.167127 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.167100    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wpbm\" (UniqueName: \"kubernetes.io/projected/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-api-access-5wpbm\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.653756 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.653716    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.656387 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.656332    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls\") pod \"success-200-isvc-46ddd-predictor-6647b68fc-zrszv\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") " pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:16.914775 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:16.914681    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:17.048779 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:17.048748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"]
Jul 01 19:43:17.052336 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:43:17.052295    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110eb4e9_8a92_4762_9350_7be4e8d78602.slice/crio-80c3f2cb19b71f68f250253adb7c83688beceb4a4faace35e9a18ccf6c1f5b4f WatchSource:0}: Error finding container 80c3f2cb19b71f68f250253adb7c83688beceb4a4faace35e9a18ccf6c1f5b4f: Status 404 returned error can't find the container with id 80c3f2cb19b71f68f250253adb7c83688beceb4a4faace35e9a18ccf6c1f5b4f
Jul 01 19:43:17.095575 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:17.095544    2566 generic.go:356] "Generic (PLEG): container finished" podID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerID="3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba" exitCode=2
Jul 01 19:43:17.095696 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:17.095610    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" event={"ID":"4fd4a47d-0c8b-4867-b55f-3a1857a88604","Type":"ContainerDied","Data":"3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba"}
Jul 01 19:43:17.096983 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:17.096958    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" event={"ID":"110eb4e9-8a92-4762-9350-7be4e8d78602","Type":"ContainerStarted","Data":"80c3f2cb19b71f68f250253adb7c83688beceb4a4faace35e9a18ccf6c1f5b4f"}
Jul 01 19:43:18.102303 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:18.102259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" event={"ID":"110eb4e9-8a92-4762-9350-7be4e8d78602","Type":"ContainerStarted","Data":"95e21dd8f3ce69ff310f7704e6d4582545325ae9d258ead1240312984e2e6b36"}
Jul 01 19:43:18.102775 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:18.102325    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:18.102775 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:18.102344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" event={"ID":"110eb4e9-8a92-4762-9350-7be4e8d78602","Type":"ContainerStarted","Data":"c8de3b2d79a31ab4c74d3ce5b9eab831a64d2e415079e3c7744aa3e62f84a448"}
Jul 01 19:43:18.102775 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:18.102398    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:18.103649 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:18.103625    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 01 19:43:18.123232 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:18.123186    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podStartSLOduration=3.123172245 podStartE2EDuration="3.123172245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:43:18.121411661 +0000 UTC m=+1194.670258036" watchObservedRunningTime="2026-07-01 19:43:18.123172245 +0000 UTC m=+1194.672018620"
Jul 01 19:43:19.105529 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.105491    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 01 19:43:19.164518 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.164483    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.18:8643/healthz\": dial tcp 10.133.0.18:8643: connect: connection refused"
Jul 01 19:43:19.169857 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.169827    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 01 19:43:19.329877 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.329846    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:43:19.373792 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.373710    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn4mz\" (UniqueName: \"kubernetes.io/projected/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-api-access-cn4mz\") pod \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") "
Jul 01 19:43:19.373792 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.373790    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-rbac-proxy-sar-config\") pod \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") "
Jul 01 19:43:19.373959 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.373912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls\") pod \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\" (UID: \"4fd4a47d-0c8b-4867-b55f-3a1857a88604\") "
Jul 01 19:43:19.374133 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.374114    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4fd4a47d-0c8b-4867-b55f-3a1857a88604" (UID: "4fd4a47d-0c8b-4867-b55f-3a1857a88604"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:43:19.375992 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.375964    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-api-access-cn4mz" (OuterVolumeSpecName: "kube-api-access-cn4mz") pod "4fd4a47d-0c8b-4867-b55f-3a1857a88604" (UID: "4fd4a47d-0c8b-4867-b55f-3a1857a88604"). InnerVolumeSpecName "kube-api-access-cn4mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:43:19.376101 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.376060    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4fd4a47d-0c8b-4867-b55f-3a1857a88604" (UID: "4fd4a47d-0c8b-4867-b55f-3a1857a88604"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:43:19.475381 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.475333    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:43:19.475543 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.475397    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd4a47d-0c8b-4867-b55f-3a1857a88604-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:43:19.475543 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:19.475408    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn4mz\" (UniqueName: \"kubernetes.io/projected/4fd4a47d-0c8b-4867-b55f-3a1857a88604-kube-api-access-cn4mz\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:43:20.110693 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.110655    2566 generic.go:356] "Generic (PLEG): container finished" podID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerID="363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d" exitCode=0
Jul 01 19:43:20.111168 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.110724    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"
Jul 01 19:43:20.111168 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.110745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" event={"ID":"4fd4a47d-0c8b-4867-b55f-3a1857a88604","Type":"ContainerDied","Data":"363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d"}
Jul 01 19:43:20.111168 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.110812    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl" event={"ID":"4fd4a47d-0c8b-4867-b55f-3a1857a88604","Type":"ContainerDied","Data":"81fd3e9881e1c50e3a973a24fe4a0344d1b0c72917394047620d95df1585a393"}
Jul 01 19:43:20.111168 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.110839    2566 scope.go:117] "RemoveContainer" containerID="3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba"
Jul 01 19:43:20.118965 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.118948    2566 scope.go:117] "RemoveContainer" containerID="363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d"
Jul 01 19:43:20.128236 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.128212    2566 scope.go:117] "RemoveContainer" containerID="3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba"
Jul 01 19:43:20.128616 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:20.128593    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba\": container with ID starting with 3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba not found: ID does not exist" containerID="3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba"
Jul 01 19:43:20.128803 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.128629    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba"} err="failed to get container status \"3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba\": rpc error: code = NotFound desc = could not find container \"3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba\": container with ID starting with 3087ed34d298e0f4b1c02998a50777854c65a5131fe8f1c6a817a98182b81cba not found: ID does not exist"
Jul 01 19:43:20.128803 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.128654    2566 scope.go:117] "RemoveContainer" containerID="363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d"
Jul 01 19:43:20.128978 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:20.128954    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d\": container with ID starting with 363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d not found: ID does not exist" containerID="363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d"
Jul 01 19:43:20.129021 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.128989    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d"} err="failed to get container status \"363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d\": rpc error: code = NotFound desc = could not find container \"363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d\": container with ID starting with 363b181fd00001da8392530f302981195eca8c5fb197365c4311498fd1ba587d not found: ID does not exist"
Jul 01 19:43:20.129479 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.129461    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"]
Jul 01 19:43:20.134149 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:20.134125    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-01ba2-predictor-547f47f477-gx5hl"]
Jul 01 19:43:21.917786 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:21.917754    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" path="/var/lib/kubelet/pods/4fd4a47d-0c8b-4867-b55f-3a1857a88604/volumes"
Jul 01 19:43:23.925203 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:23.925175    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:43:23.925883 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:23.925866    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:43:24.110492 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:24.110467    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:43:24.110879 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:24.110853    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 01 19:43:26.457773 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.457739    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"]
Jul 01 19:43:26.458255 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.458030    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container" containerID="cri-o://4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7" gracePeriod=30
Jul 01 19:43:26.458255 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.458086    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kube-rbac-proxy" containerID="cri-o://ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a" gracePeriod=30
Jul 01 19:43:26.485165 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485129    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"]
Jul 01 19:43:26.485422 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485409    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container"
Jul 01 19:43:26.485466 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485424    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container"
Jul 01 19:43:26.485466 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485434    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kube-rbac-proxy"
Jul 01 19:43:26.485466 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485440    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kube-rbac-proxy"
Jul 01 19:43:26.485561 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485481    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kserve-container"
Jul 01 19:43:26.485561 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.485490    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd4a47d-0c8b-4867-b55f-3a1857a88604" containerName="kube-rbac-proxy"
Jul 01 19:43:26.489688 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.489668    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.492540 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.492510    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c6a40-predictor-serving-cert\""
Jul 01 19:43:26.492686 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.492555    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c6a40-kube-rbac-proxy-sar-config\""
Jul 01 19:43:26.500002 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.499982    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"]
Jul 01 19:43:26.633079 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.633041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.633269 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.633133    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj9c9\" (UniqueName: \"kubernetes.io/projected/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-api-access-gj9c9\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.633269 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.633167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.733650 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.733555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.733650 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.733648    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj9c9\" (UniqueName: \"kubernetes.io/projected/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-api-access-gj9c9\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.733878 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.733680    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.733878 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:26.733723    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-serving-cert: secret "success-200-isvc-c6a40-predictor-serving-cert" not found
Jul 01 19:43:26.733878 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:26.733811    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls podName:7f7ebb07-09db-4810-ac21-cb4ff6867654 nodeName:}" failed. No retries permitted until 2026-07-01 19:43:27.23378833 +0000 UTC m=+1203.782634684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls") pod "success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" (UID: "7f7ebb07-09db-4810-ac21-cb4ff6867654") : secret "success-200-isvc-c6a40-predictor-serving-cert" not found
Jul 01 19:43:26.734805 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.734783    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:26.744178 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:26.744147    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj9c9\" (UniqueName: \"kubernetes.io/projected/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-api-access-gj9c9\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:27.136765 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:27.136731    2566 generic.go:356] "Generic (PLEG): container finished" podID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerID="ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a" exitCode=2
Jul 01 19:43:27.136946 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:27.136801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" event={"ID":"e9acb56c-4c2e-48ff-9baf-86c9f9428d78","Type":"ContainerDied","Data":"ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a"}
Jul 01 19:43:27.237223 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:27.237182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:27.239655 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:27.239629    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls\") pod \"success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") " pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:27.400724 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:27.400634    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:27.539038 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:27.539008    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"]
Jul 01 19:43:27.542015 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:43:27.541982    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7ebb07_09db_4810_ac21_cb4ff6867654.slice/crio-d51f00f6f6d490fb7a7c707f3558df322ccbb0a15c62be8d374db932404a8c30 WatchSource:0}: Error finding container d51f00f6f6d490fb7a7c707f3558df322ccbb0a15c62be8d374db932404a8c30: Status 404 returned error can't find the container with id d51f00f6f6d490fb7a7c707f3558df322ccbb0a15c62be8d374db932404a8c30
Jul 01 19:43:28.142410 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.142357    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" event={"ID":"7f7ebb07-09db-4810-ac21-cb4ff6867654","Type":"ContainerStarted","Data":"eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512"}
Jul 01 19:43:28.142622 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.142441    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" event={"ID":"7f7ebb07-09db-4810-ac21-cb4ff6867654","Type":"ContainerStarted","Data":"62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4"}
Jul 01 19:43:28.142622 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.142467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" event={"ID":"7f7ebb07-09db-4810-ac21-cb4ff6867654","Type":"ContainerStarted","Data":"d51f00f6f6d490fb7a7c707f3558df322ccbb0a15c62be8d374db932404a8c30"}
Jul 01 19:43:28.142622 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.142501    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:28.142769 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.142630    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:28.143970 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.143945    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:43:28.165181 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.165131    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podStartSLOduration=2.165115273 podStartE2EDuration="2.165115273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:43:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:43:28.164055414 +0000 UTC m=+1204.712901791" watchObservedRunningTime="2026-07-01 19:43:28.165115273 +0000 UTC m=+1204.713961649"
Jul 01 19:43:28.894835 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:28.894790    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.19:8643/healthz\": dial tcp 10.133.0.19:8643: connect: connection refused"
Jul 01 19:43:29.147148 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.147059    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:43:29.899417 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.899392    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:43:29.957391 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.957325    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v5kt\" (UniqueName: \"kubernetes.io/projected/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-api-access-8v5kt\") pod \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") "
Jul 01 19:43:29.957563 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.957454    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-proxy-tls\") pod \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") "
Jul 01 19:43:29.957563 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.957526    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-rbac-proxy-sar-config\") pod \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\" (UID: \"e9acb56c-4c2e-48ff-9baf-86c9f9428d78\") "
Jul 01 19:43:29.957934 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.957892    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e9acb56c-4c2e-48ff-9baf-86c9f9428d78" (UID: "e9acb56c-4c2e-48ff-9baf-86c9f9428d78"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:43:29.959560 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.959537    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-api-access-8v5kt" (OuterVolumeSpecName: "kube-api-access-8v5kt") pod "e9acb56c-4c2e-48ff-9baf-86c9f9428d78" (UID: "e9acb56c-4c2e-48ff-9baf-86c9f9428d78"). InnerVolumeSpecName "kube-api-access-8v5kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:43:29.959757 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:29.959738    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e9acb56c-4c2e-48ff-9baf-86c9f9428d78" (UID: "e9acb56c-4c2e-48ff-9baf-86c9f9428d78"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:43:30.058441 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.058408    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:43:30.058441 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.058437    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v5kt\" (UniqueName: \"kubernetes.io/projected/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-kube-api-access-8v5kt\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:43:30.058611 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.058447    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9acb56c-4c2e-48ff-9baf-86c9f9428d78-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:43:30.151840 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.151807    2566 generic.go:356] "Generic (PLEG): container finished" podID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerID="4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7" exitCode=0
Jul 01 19:43:30.152013 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.151863    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"
Jul 01 19:43:30.152013 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.151890    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" event={"ID":"e9acb56c-4c2e-48ff-9baf-86c9f9428d78","Type":"ContainerDied","Data":"4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7"}
Jul 01 19:43:30.152013 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.151958    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz" event={"ID":"e9acb56c-4c2e-48ff-9baf-86c9f9428d78","Type":"ContainerDied","Data":"f59e814ccbde7595865013b29afdc35d0b68614c802a15e575a71ef4399e0d17"}
Jul 01 19:43:30.152013 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.151997    2566 scope.go:117] "RemoveContainer" containerID="ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a"
Jul 01 19:43:30.161096 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.161075    2566 scope.go:117] "RemoveContainer" containerID="4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7"
Jul 01 19:43:30.168324 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.168309    2566 scope.go:117] "RemoveContainer" containerID="ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a"
Jul 01 19:43:30.168614 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:30.168597    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a\": container with ID starting with ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a not found: ID does not exist" containerID="ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a"
Jul 01 19:43:30.168663 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.168626    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a"} err="failed to get container status \"ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a\": rpc error: code = NotFound desc = could not find container \"ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a\": container with ID starting with ecf34169bea6d13ff761f2d56b5357224d8ad6a6e686bd96a15546603899ad7a not found: ID does not exist"
Jul 01 19:43:30.168663 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.168642    2566 scope.go:117] "RemoveContainer" containerID="4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7"
Jul 01 19:43:30.168840 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:43:30.168822    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7\": container with ID starting with 4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7 not found: ID does not exist" containerID="4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7"
Jul 01 19:43:30.168896 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.168848    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7"} err="failed to get container status \"4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7\": rpc error: code = NotFound desc = could not find container \"4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7\": container with ID starting with 4d356a4ba0d6be9234da75e418b59ffc99ed18f59e6daee77d08b9e5ceba15c7 not found: ID does not exist"
Jul 01 19:43:30.174427 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.174406    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"]
Jul 01 19:43:30.176873 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:30.176853    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ac147-predictor-6cd58fc96f-b2hmz"]
Jul 01 19:43:31.917344 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:31.917306    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" path="/var/lib/kubelet/pods/e9acb56c-4c2e-48ff-9baf-86c9f9428d78/volumes"
Jul 01 19:43:34.111275 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:34.111229    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 01 19:43:34.152066 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:34.152039    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:43:34.152649 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:34.152617    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:43:44.111736 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:44.111695    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 01 19:43:44.152533 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:44.152494    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:43:54.110973 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:54.110933    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 01 19:43:54.153182 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:43:54.153136    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:44:04.111537 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:04.111506    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:44:04.152607 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:04.152538    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:44:14.153180 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:14.153107    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:44:26.238450 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.238383    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"]
Jul 01 19:44:26.238978 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.238686    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container" containerID="cri-o://c8de3b2d79a31ab4c74d3ce5b9eab831a64d2e415079e3c7744aa3e62f84a448" gracePeriod=30
Jul 01 19:44:26.239063 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.238755    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kube-rbac-proxy" containerID="cri-o://95e21dd8f3ce69ff310f7704e6d4582545325ae9d258ead1240312984e2e6b36" gracePeriod=30
Jul 01 19:44:26.241098 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241072    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"]
Jul 01 19:44:26.241356 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241341    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container"
Jul 01 19:44:26.241356 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241356    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container"
Jul 01 19:44:26.241511 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241388    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kube-rbac-proxy"
Jul 01 19:44:26.241511 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241396    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kube-rbac-proxy"
Jul 01 19:44:26.241511 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241454    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kserve-container"
Jul 01 19:44:26.241511 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.241461    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9acb56c-4c2e-48ff-9baf-86c9f9428d78" containerName="kube-rbac-proxy"
Jul 01 19:44:26.245519 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.245502    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.247910 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.247891    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-88843-predictor-serving-cert\""
Jul 01 19:44:26.248128 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.248109    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-88843-kube-rbac-proxy-sar-config\""
Jul 01 19:44:26.255493 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.255470    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"]
Jul 01 19:44:26.289490 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.289461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-proxy-tls\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.289638 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.289502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.289638 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.289611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhcc\" (UniqueName: \"kubernetes.io/projected/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-api-access-czhcc\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.390402 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.390347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-proxy-tls\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.390402 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.390403    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.390648 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.390452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhcc\" (UniqueName: \"kubernetes.io/projected/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-api-access-czhcc\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.391206 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.391181    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.392873 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.392852    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-proxy-tls\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.399328 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.399306    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhcc\" (UniqueName: \"kubernetes.io/projected/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-api-access-czhcc\") pod \"success-200-isvc-88843-predictor-694fdcfc59-krdsf\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") " pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.563517 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.563489    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:26.698147 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:26.697776    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"]
Jul 01 19:44:27.360443 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.360408    2566 generic.go:356] "Generic (PLEG): container finished" podID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerID="95e21dd8f3ce69ff310f7704e6d4582545325ae9d258ead1240312984e2e6b36" exitCode=2
Jul 01 19:44:27.360874 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.360480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" event={"ID":"110eb4e9-8a92-4762-9350-7be4e8d78602","Type":"ContainerDied","Data":"95e21dd8f3ce69ff310f7704e6d4582545325ae9d258ead1240312984e2e6b36"}
Jul 01 19:44:27.362138 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.362114    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" event={"ID":"13b71d55-6000-452c-8ebd-d8a92ac9b6e7","Type":"ContainerStarted","Data":"96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1"}
Jul 01 19:44:27.362260 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.362171    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" event={"ID":"13b71d55-6000-452c-8ebd-d8a92ac9b6e7","Type":"ContainerStarted","Data":"ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea"}
Jul 01 19:44:27.362260 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.362201    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" event={"ID":"13b71d55-6000-452c-8ebd-d8a92ac9b6e7","Type":"ContainerStarted","Data":"19b0a9bad871c572d8b06bef34b2e7639cef4099893edae0bbe35e2d99ccac8c"}
Jul 01 19:44:27.362260 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.362246    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:27.382799 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:27.382737    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podStartSLOduration=1.3827216039999999 podStartE2EDuration="1.382721604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:44:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:44:27.381535876 +0000 UTC m=+1263.930382449" watchObservedRunningTime="2026-07-01 19:44:27.382721604 +0000 UTC m=+1263.931567979"
Jul 01 19:44:28.365984 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:28.365953    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:28.367212 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:28.367187    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:44:29.105923 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.105876    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused"
Jul 01 19:44:29.371654 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.371622    2566 generic.go:356] "Generic (PLEG): container finished" podID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerID="c8de3b2d79a31ab4c74d3ce5b9eab831a64d2e415079e3c7744aa3e62f84a448" exitCode=0
Jul 01 19:44:29.372078 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.371694    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" event={"ID":"110eb4e9-8a92-4762-9350-7be4e8d78602","Type":"ContainerDied","Data":"c8de3b2d79a31ab4c74d3ce5b9eab831a64d2e415079e3c7744aa3e62f84a448"}
Jul 01 19:44:29.372179 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.372155    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:44:29.490059 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.490029    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:44:29.514680 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.514654    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wpbm\" (UniqueName: \"kubernetes.io/projected/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-api-access-5wpbm\") pod \"110eb4e9-8a92-4762-9350-7be4e8d78602\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") "
Jul 01 19:44:29.514828 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.514686    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-rbac-proxy-sar-config\") pod \"110eb4e9-8a92-4762-9350-7be4e8d78602\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") "
Jul 01 19:44:29.514828 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.514724    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls\") pod \"110eb4e9-8a92-4762-9350-7be4e8d78602\" (UID: \"110eb4e9-8a92-4762-9350-7be4e8d78602\") "
Jul 01 19:44:29.515066 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.515038    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "110eb4e9-8a92-4762-9350-7be4e8d78602" (UID: "110eb4e9-8a92-4762-9350-7be4e8d78602"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:44:29.516976 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.516943    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "110eb4e9-8a92-4762-9350-7be4e8d78602" (UID: "110eb4e9-8a92-4762-9350-7be4e8d78602"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:44:29.517095 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.517029    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-api-access-5wpbm" (OuterVolumeSpecName: "kube-api-access-5wpbm") pod "110eb4e9-8a92-4762-9350-7be4e8d78602" (UID: "110eb4e9-8a92-4762-9350-7be4e8d78602"). InnerVolumeSpecName "kube-api-access-5wpbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:44:29.615667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.615564    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wpbm\" (UniqueName: \"kubernetes.io/projected/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-api-access-5wpbm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:44:29.615667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.615599    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/110eb4e9-8a92-4762-9350-7be4e8d78602-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:44:29.615667 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:29.615610    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/110eb4e9-8a92-4762-9350-7be4e8d78602-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:44:30.376357 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:30.376314    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv" event={"ID":"110eb4e9-8a92-4762-9350-7be4e8d78602","Type":"ContainerDied","Data":"80c3f2cb19b71f68f250253adb7c83688beceb4a4faace35e9a18ccf6c1f5b4f"}
Jul 01 19:44:30.376801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:30.376387    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"
Jul 01 19:44:30.376801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:30.376415    2566 scope.go:117] "RemoveContainer" containerID="95e21dd8f3ce69ff310f7704e6d4582545325ae9d258ead1240312984e2e6b36"
Jul 01 19:44:30.384400 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:30.384361    2566 scope.go:117] "RemoveContainer" containerID="c8de3b2d79a31ab4c74d3ce5b9eab831a64d2e415079e3c7744aa3e62f84a448"
Jul 01 19:44:30.394805 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:30.394784    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"]
Jul 01 19:44:30.400865 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:30.400842    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-46ddd-predictor-6647b68fc-zrszv"]
Jul 01 19:44:31.917263 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:31.917232    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" path="/var/lib/kubelet/pods/110eb4e9-8a92-4762-9350-7be4e8d78602/volumes"
Jul 01 19:44:34.376928 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:34.376900    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:44:34.377351 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:34.377326    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:44:44.377668 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:44.377617    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:44:54.377776 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:44:54.377733    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:45:04.378329 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:45:04.378287    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:45:14.378283 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:45:14.378251    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:48:23.945184 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:48:23.945154    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:48:23.945811 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:48:23.945520    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:52:41.291423 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291391    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"]
Jul 01 19:52:41.293891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291677    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container"
Jul 01 19:52:41.293891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291689    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container"
Jul 01 19:52:41.293891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291710    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kube-rbac-proxy"
Jul 01 19:52:41.293891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291718    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kube-rbac-proxy"
Jul 01 19:52:41.293891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291769    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kube-rbac-proxy"
Jul 01 19:52:41.293891 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.291779    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="110eb4e9-8a92-4762-9350-7be4e8d78602" containerName="kserve-container"
Jul 01 19:52:41.294786 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.294769    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.297427 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.297404    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-74731-predictor-serving-cert\""
Jul 01 19:52:41.297552 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.297438    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-74731-kube-rbac-proxy-sar-config\""
Jul 01 19:52:41.309557 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.309526    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"]
Jul 01 19:52:41.317400 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.317353    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"]
Jul 01 19:52:41.317665 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.317610    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" containerID="cri-o://62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4" gracePeriod=30
Jul 01 19:52:41.317772 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.317724    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kube-rbac-proxy" containerID="cri-o://eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512" gracePeriod=30
Jul 01 19:52:41.413801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.413766    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w498\" (UniqueName: \"kubernetes.io/projected/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-api-access-7w498\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.413974 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.413832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-proxy-tls\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.413974 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.413869    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.515305 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.515260    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w498\" (UniqueName: \"kubernetes.io/projected/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-api-access-7w498\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.515518 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.515315    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-proxy-tls\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.515518 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.515346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.516125 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.516090    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.517907 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.517883    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-proxy-tls\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.524465 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.524444    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w498\" (UniqueName: \"kubernetes.io/projected/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-api-access-7w498\") pod \"success-200-isvc-74731-predictor-ddb44967c-gr2gj\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") " pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.606243 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.606153    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:41.743924 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:41.743899    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"]
Jul 01 19:52:41.745911 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:52:41.745883    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11a248b_5d39_4a65_9a5c_9b7bff577cd8.slice/crio-770422b229afb7b4a9641d381acdd0581803b123dd80bf826047cf2fb2f1f478 WatchSource:0}: Error finding container 770422b229afb7b4a9641d381acdd0581803b123dd80bf826047cf2fb2f1f478: Status 404 returned error can't find the container with id 770422b229afb7b4a9641d381acdd0581803b123dd80bf826047cf2fb2f1f478
Jul 01 19:52:42.070654 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.070622    2566 generic.go:356] "Generic (PLEG): container finished" podID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerID="eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512" exitCode=2
Jul 01 19:52:42.070842 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.070672    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" event={"ID":"7f7ebb07-09db-4810-ac21-cb4ff6867654","Type":"ContainerDied","Data":"eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512"}
Jul 01 19:52:42.072465 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.072442    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" event={"ID":"c11a248b-5d39-4a65-9a5c-9b7bff577cd8","Type":"ContainerStarted","Data":"5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef"}
Jul 01 19:52:42.072608 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.072486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" event={"ID":"c11a248b-5d39-4a65-9a5c-9b7bff577cd8","Type":"ContainerStarted","Data":"7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc"}
Jul 01 19:52:42.072608 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.072506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" event={"ID":"c11a248b-5d39-4a65-9a5c-9b7bff577cd8","Type":"ContainerStarted","Data":"770422b229afb7b4a9641d381acdd0581803b123dd80bf826047cf2fb2f1f478"}
Jul 01 19:52:42.072608 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.072576    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:42.090856 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:42.090797    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podStartSLOduration=1.090780348 podStartE2EDuration="1.090780348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:52:42.090350784 +0000 UTC m=+1758.639197171" watchObservedRunningTime="2026-07-01 19:52:42.090780348 +0000 UTC m=+1758.639626722"
Jul 01 19:52:43.076412 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:43.076378    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:43.077594 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:43.077564    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 01 19:52:44.079909 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.079866    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 01 19:52:44.147919 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.147881    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 01 19:52:44.153442 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.153407    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 01 19:52:44.568777 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.568752    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:52:44.640114 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.640041    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj9c9\" (UniqueName: \"kubernetes.io/projected/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-api-access-gj9c9\") pod \"7f7ebb07-09db-4810-ac21-cb4ff6867654\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") "
Jul 01 19:52:44.640114 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.640084    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-rbac-proxy-sar-config\") pod \"7f7ebb07-09db-4810-ac21-cb4ff6867654\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") "
Jul 01 19:52:44.640294 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.640208    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls\") pod \"7f7ebb07-09db-4810-ac21-cb4ff6867654\" (UID: \"7f7ebb07-09db-4810-ac21-cb4ff6867654\") "
Jul 01 19:52:44.640485 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.640455    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7f7ebb07-09db-4810-ac21-cb4ff6867654" (UID: "7f7ebb07-09db-4810-ac21-cb4ff6867654"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:52:44.642309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.642286    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7f7ebb07-09db-4810-ac21-cb4ff6867654" (UID: "7f7ebb07-09db-4810-ac21-cb4ff6867654"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:52:44.642309 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.642293    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-api-access-gj9c9" (OuterVolumeSpecName: "kube-api-access-gj9c9") pod "7f7ebb07-09db-4810-ac21-cb4ff6867654" (UID: "7f7ebb07-09db-4810-ac21-cb4ff6867654"). InnerVolumeSpecName "kube-api-access-gj9c9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:52:44.741414 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.741379    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj9c9\" (UniqueName: \"kubernetes.io/projected/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-api-access-gj9c9\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:52:44.741414 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.741406    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7f7ebb07-09db-4810-ac21-cb4ff6867654-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:52:44.741414 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:44.741417    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f7ebb07-09db-4810-ac21-cb4ff6867654-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:52:45.084330 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.084290    2566 generic.go:356] "Generic (PLEG): container finished" podID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerID="62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4" exitCode=0
Jul 01 19:52:45.084848 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.084338    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"
Jul 01 19:52:45.084848 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.084391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" event={"ID":"7f7ebb07-09db-4810-ac21-cb4ff6867654","Type":"ContainerDied","Data":"62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4"}
Jul 01 19:52:45.084848 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.084471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh" event={"ID":"7f7ebb07-09db-4810-ac21-cb4ff6867654","Type":"ContainerDied","Data":"d51f00f6f6d490fb7a7c707f3558df322ccbb0a15c62be8d374db932404a8c30"}
Jul 01 19:52:45.084848 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.084518    2566 scope.go:117] "RemoveContainer" containerID="eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512"
Jul 01 19:52:45.093331 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.093311    2566 scope.go:117] "RemoveContainer" containerID="62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4"
Jul 01 19:52:45.101004 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.100980    2566 scope.go:117] "RemoveContainer" containerID="eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512"
Jul 01 19:52:45.101351 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:52:45.101331    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512\": container with ID starting with eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512 not found: ID does not exist" containerID="eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512"
Jul 01 19:52:45.101440 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.101361    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512"} err="failed to get container status \"eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512\": rpc error: code = NotFound desc = could not find container \"eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512\": container with ID starting with eba4cdf3f99eb331427e03d60ab6936eaa36f9ef4c4a36d2d1be2611c58ff512 not found: ID does not exist"
Jul 01 19:52:45.101440 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.101403    2566 scope.go:117] "RemoveContainer" containerID="62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4"
Jul 01 19:52:45.101699 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:52:45.101680    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4\": container with ID starting with 62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4 not found: ID does not exist" containerID="62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4"
Jul 01 19:52:45.101753 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.101711    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4"} err="failed to get container status \"62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4\": rpc error: code = NotFound desc = could not find container \"62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4\": container with ID starting with 62a434c63abeaa7f35709686e2704d75355d21a2695fb837a892cae986dfd4b4 not found: ID does not exist"
Jul 01 19:52:45.105908 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.105880    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"]
Jul 01 19:52:45.108595 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.108572    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6a40-predictor-759bb4bdbf-7p8wh"]
Jul 01 19:52:45.918035 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:45.917997    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" path="/var/lib/kubelet/pods/7f7ebb07-09db-4810-ac21-cb4ff6867654/volumes"
Jul 01 19:52:49.084274 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:49.084240    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:52:49.084843 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:49.084816    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 01 19:52:59.084931 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:52:59.084888    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 01 19:53:09.085197 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:09.085148    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 01 19:53:19.084788 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:19.084700    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 01 19:53:23.965919 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:23.965891    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:53:23.967187 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:23.967167    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:53:29.085518 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:29.085481    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:53:41.211075 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211038    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"]
Jul 01 19:53:41.211523 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211332    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container"
Jul 01 19:53:41.211523 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211345    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container"
Jul 01 19:53:41.211523 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211355    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kube-rbac-proxy"
Jul 01 19:53:41.211523 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211360    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kube-rbac-proxy"
Jul 01 19:53:41.211523 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211440    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kserve-container"
Jul 01 19:53:41.211523 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.211450    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f7ebb07-09db-4810-ac21-cb4ff6867654" containerName="kube-rbac-proxy"
Jul 01 19:53:41.214340 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.214320    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.217064 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.217043    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ee34f-kube-rbac-proxy-sar-config\""
Jul 01 19:53:41.217311 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.217295    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ee34f-predictor-serving-cert\""
Jul 01 19:53:41.223655 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.223635    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"]
Jul 01 19:53:41.223844 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.223823    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" containerID="cri-o://ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea" gracePeriod=30
Jul 01 19:53:41.223905 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.223886    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kube-rbac-proxy" containerID="cri-o://96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1" gracePeriod=30
Jul 01 19:53:41.233146 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.233122    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"]
Jul 01 19:53:41.246037 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.246015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.246152 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.246049    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2bcq\" (UniqueName: \"kubernetes.io/projected/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-api-access-q2bcq\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.246152 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.246082    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-proxy-tls\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.347455 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.347355    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.347455 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.347441    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2bcq\" (UniqueName: \"kubernetes.io/projected/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-api-access-q2bcq\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.347662 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.347515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-proxy-tls\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.348129 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.348103    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.349910 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.349890    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-proxy-tls\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.359269 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.359241    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2bcq\" (UniqueName: \"kubernetes.io/projected/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-api-access-q2bcq\") pod \"success-200-isvc-ee34f-predictor-6bdb89f555-k5d95\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") " pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.524857 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.524768    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:41.671693 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:41.671660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"]
Jul 01 19:53:41.675653 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:53:41.675627    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07cd3f38_e097_4fe2_b446_f6875dbd0ab9.slice/crio-b861ed400baef7160200d708a56e62369c9f05aa37051570bb29940a364b2c93 WatchSource:0}: Error finding container b861ed400baef7160200d708a56e62369c9f05aa37051570bb29940a364b2c93: Status 404 returned error can't find the container with id b861ed400baef7160200d708a56e62369c9f05aa37051570bb29940a364b2c93
Jul 01 19:53:42.291108 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.291066    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" event={"ID":"07cd3f38-e097-4fe2-b446-f6875dbd0ab9","Type":"ContainerStarted","Data":"187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e"}
Jul 01 19:53:42.291582 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.291140    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:42.291582 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.291214    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:42.291582 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.291277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" event={"ID":"07cd3f38-e097-4fe2-b446-f6875dbd0ab9","Type":"ContainerStarted","Data":"3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482"}
Jul 01 19:53:42.291582 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.291320    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" event={"ID":"07cd3f38-e097-4fe2-b446-f6875dbd0ab9","Type":"ContainerStarted","Data":"b861ed400baef7160200d708a56e62369c9f05aa37051570bb29940a364b2c93"}
Jul 01 19:53:42.292960 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.292936    2566 generic.go:356] "Generic (PLEG): container finished" podID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerID="96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1" exitCode=2
Jul 01 19:53:42.293083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.292972    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" event={"ID":"13b71d55-6000-452c-8ebd-d8a92ac9b6e7","Type":"ContainerDied","Data":"96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1"}
Jul 01 19:53:42.293083 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.292969    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 01 19:53:42.308266 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:42.308223    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podStartSLOduration=1.308209495 podStartE2EDuration="1.308209495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:53:42.306734357 +0000 UTC m=+1818.855580733" watchObservedRunningTime="2026-07-01 19:53:42.308209495 +0000 UTC m=+1818.857055870"
Jul 01 19:53:43.297558 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:43.297516    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 01 19:53:44.373016 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.372974    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused"
Jul 01 19:53:44.377381 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.377333    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 01 19:53:44.474642 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.474617    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:53:44.575917 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.575879    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-proxy-tls\") pod \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") "
Jul 01 19:53:44.576085 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.575967    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czhcc\" (UniqueName: \"kubernetes.io/projected/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-api-access-czhcc\") pod \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") "
Jul 01 19:53:44.576085 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.576054    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-rbac-proxy-sar-config\") pod \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\" (UID: \"13b71d55-6000-452c-8ebd-d8a92ac9b6e7\") "
Jul 01 19:53:44.576421 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.576387    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "13b71d55-6000-452c-8ebd-d8a92ac9b6e7" (UID: "13b71d55-6000-452c-8ebd-d8a92ac9b6e7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:53:44.578111 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.578090    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "13b71d55-6000-452c-8ebd-d8a92ac9b6e7" (UID: "13b71d55-6000-452c-8ebd-d8a92ac9b6e7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:53:44.578194 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.578157    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-api-access-czhcc" (OuterVolumeSpecName: "kube-api-access-czhcc") pod "13b71d55-6000-452c-8ebd-d8a92ac9b6e7" (UID: "13b71d55-6000-452c-8ebd-d8a92ac9b6e7"). InnerVolumeSpecName "kube-api-access-czhcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:53:44.677481 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.677446    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czhcc\" (UniqueName: \"kubernetes.io/projected/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-api-access-czhcc\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:53:44.677481 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.677475    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:53:44.677481 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:44.677485    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/13b71d55-6000-452c-8ebd-d8a92ac9b6e7-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:53:45.309130 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.309097    2566 generic.go:356] "Generic (PLEG): container finished" podID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerID="ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea" exitCode=0
Jul 01 19:53:45.309297 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.309150    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"
Jul 01 19:53:45.309297 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.309172    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" event={"ID":"13b71d55-6000-452c-8ebd-d8a92ac9b6e7","Type":"ContainerDied","Data":"ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea"}
Jul 01 19:53:45.309297 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.309228    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf" event={"ID":"13b71d55-6000-452c-8ebd-d8a92ac9b6e7","Type":"ContainerDied","Data":"19b0a9bad871c572d8b06bef34b2e7639cef4099893edae0bbe35e2d99ccac8c"}
Jul 01 19:53:45.309297 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.309254    2566 scope.go:117] "RemoveContainer" containerID="96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1"
Jul 01 19:53:45.317540 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.317524    2566 scope.go:117] "RemoveContainer" containerID="ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea"
Jul 01 19:53:45.324612 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.324594    2566 scope.go:117] "RemoveContainer" containerID="96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1"
Jul 01 19:53:45.324867 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:53:45.324846    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1\": container with ID starting with 96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1 not found: ID does not exist" containerID="96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1"
Jul 01 19:53:45.324918 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.324875    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1"} err="failed to get container status \"96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1\": rpc error: code = NotFound desc = could not find container \"96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1\": container with ID starting with 96009e8288e8ab5178d925beb5c406671ed08b3d4881a1ee9cee151bc5566ce1 not found: ID does not exist"
Jul 01 19:53:45.324918 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.324891    2566 scope.go:117] "RemoveContainer" containerID="ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea"
Jul 01 19:53:45.325098 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:53:45.325081    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea\": container with ID starting with ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea not found: ID does not exist" containerID="ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea"
Jul 01 19:53:45.325147 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.325107    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea"} err="failed to get container status \"ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea\": rpc error: code = NotFound desc = could not find container \"ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea\": container with ID starting with ecfdf6b9bde47de6a3ecc1113491c6aa19f46889f88b56d08e0aaa681cdf2bea not found: ID does not exist"
Jul 01 19:53:45.331607 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.331587    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"]
Jul 01 19:53:45.338058 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.338034    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-88843-predictor-694fdcfc59-krdsf"]
Jul 01 19:53:45.917312 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:45.917279    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" path="/var/lib/kubelet/pods/13b71d55-6000-452c-8ebd-d8a92ac9b6e7/volumes"
Jul 01 19:53:48.302110 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:48.302079    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:53:48.302716 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:48.302688    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 01 19:53:51.617728 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.617695    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"]
Jul 01 19:53:51.618173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.617945    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container"
Jul 01 19:53:51.618173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.617955    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container"
Jul 01 19:53:51.618173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.617975    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kube-rbac-proxy"
Jul 01 19:53:51.618173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.617981    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kube-rbac-proxy"
Jul 01 19:53:51.618173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.618025    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kube-rbac-proxy"
Jul 01 19:53:51.618173 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.618034    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b71d55-6000-452c-8ebd-d8a92ac9b6e7" containerName="kserve-container"
Jul 01 19:53:51.621033 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.621009    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.623799 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.623776    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b4182-kube-rbac-proxy-sar-config\""
Jul 01 19:53:51.623904 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.623775    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b4182-predictor-serving-cert\""
Jul 01 19:53:51.633699 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.633674    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"]
Jul 01 19:53:51.728533 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.728499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lmh\" (UniqueName: \"kubernetes.io/projected/0e0a9972-6658-46fe-b0ce-20604af45554-kube-api-access-p9lmh\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.728729 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.728559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e0a9972-6658-46fe-b0ce-20604af45554-proxy-tls\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.728729 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.728628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0e0a9972-6658-46fe-b0ce-20604af45554-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.803034 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.802994    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"]
Jul 01 19:53:51.803313 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.803270    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container" containerID="cri-o://7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc" gracePeriod=30
Jul 01 19:53:51.803419 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.803290    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kube-rbac-proxy" containerID="cri-o://5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef" gracePeriod=30
Jul 01 19:53:51.829823 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.829791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lmh\" (UniqueName: \"kubernetes.io/projected/0e0a9972-6658-46fe-b0ce-20604af45554-kube-api-access-p9lmh\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.829949 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.829838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e0a9972-6658-46fe-b0ce-20604af45554-proxy-tls\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.829949 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.829864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0e0a9972-6658-46fe-b0ce-20604af45554-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.830428 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.830409    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0e0a9972-6658-46fe-b0ce-20604af45554-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.832227 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.832209    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e0a9972-6658-46fe-b0ce-20604af45554-proxy-tls\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.839202 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.839175    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lmh\" (UniqueName: \"kubernetes.io/projected/0e0a9972-6658-46fe-b0ce-20604af45554-kube-api-access-p9lmh\") pod \"success-200-isvc-b4182-predictor-58995c77cf-8brpg\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") " pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:51.930817 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:51.930781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:52.071176 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.071144    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"]
Jul 01 19:53:52.074433 ip-10-0-129-40 kubenswrapper[2566]: W0701 19:53:52.074405    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0a9972_6658_46fe_b0ce_20604af45554.slice/crio-9b3b02f330a95dfa3dbea6821074f735678f5ce0be4eb6594e9076c659378be7 WatchSource:0}: Error finding container 9b3b02f330a95dfa3dbea6821074f735678f5ce0be4eb6594e9076c659378be7: Status 404 returned error can't find the container with id 9b3b02f330a95dfa3dbea6821074f735678f5ce0be4eb6594e9076c659378be7
Jul 01 19:53:52.334456 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.334421    2566 generic.go:356] "Generic (PLEG): container finished" podID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerID="5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef" exitCode=2
Jul 01 19:53:52.334637 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.334496    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" event={"ID":"c11a248b-5d39-4a65-9a5c-9b7bff577cd8","Type":"ContainerDied","Data":"5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef"}
Jul 01 19:53:52.336238 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.336221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" event={"ID":"0e0a9972-6658-46fe-b0ce-20604af45554","Type":"ContainerStarted","Data":"beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5"}
Jul 01 19:53:52.336291 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.336263    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" event={"ID":"0e0a9972-6658-46fe-b0ce-20604af45554","Type":"ContainerStarted","Data":"c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc"}
Jul 01 19:53:52.336291 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.336285    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" event={"ID":"0e0a9972-6658-46fe-b0ce-20604af45554","Type":"ContainerStarted","Data":"9b3b02f330a95dfa3dbea6821074f735678f5ce0be4eb6594e9076c659378be7"}
Jul 01 19:53:52.336357 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.336308    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:52.336418 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.336357    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:52.337781 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.337755    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 19:53:52.355873 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:52.355808    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podStartSLOduration=1.355789033 podStartE2EDuration="1.355789033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:53:52.355246531 +0000 UTC m=+1828.904092907" watchObservedRunningTime="2026-07-01 19:53:52.355789033 +0000 UTC m=+1828.904635409"
Jul 01 19:53:53.339868 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:53.339831    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 19:53:54.080688 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:54.080634    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused"
Jul 01 19:53:55.047966 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.047938    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:53:55.158058 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.157952    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w498\" (UniqueName: \"kubernetes.io/projected/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-api-access-7w498\") pod \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") "
Jul 01 19:53:55.158058 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.158015    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-proxy-tls\") pod \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") "
Jul 01 19:53:55.158284 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.158093    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-rbac-proxy-sar-config\") pod \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\" (UID: \"c11a248b-5d39-4a65-9a5c-9b7bff577cd8\") "
Jul 01 19:53:55.158506 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.158480    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c11a248b-5d39-4a65-9a5c-9b7bff577cd8" (UID: "c11a248b-5d39-4a65-9a5c-9b7bff577cd8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:53:55.160214 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.160195    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-api-access-7w498" (OuterVolumeSpecName: "kube-api-access-7w498") pod "c11a248b-5d39-4a65-9a5c-9b7bff577cd8" (UID: "c11a248b-5d39-4a65-9a5c-9b7bff577cd8"). InnerVolumeSpecName "kube-api-access-7w498". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:53:55.160271 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.160261    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c11a248b-5d39-4a65-9a5c-9b7bff577cd8" (UID: "c11a248b-5d39-4a65-9a5c-9b7bff577cd8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:53:55.258811 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.258773    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:53:55.258811 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.258802    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w498\" (UniqueName: \"kubernetes.io/projected/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-kube-api-access-7w498\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:53:55.258811 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.258812    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c11a248b-5d39-4a65-9a5c-9b7bff577cd8-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 19:53:55.347216 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.347183    2566 generic.go:356] "Generic (PLEG): container finished" podID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerID="7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc" exitCode=0
Jul 01 19:53:55.347426 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.347235    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"
Jul 01 19:53:55.347426 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.347251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" event={"ID":"c11a248b-5d39-4a65-9a5c-9b7bff577cd8","Type":"ContainerDied","Data":"7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc"}
Jul 01 19:53:55.347426 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.347313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj" event={"ID":"c11a248b-5d39-4a65-9a5c-9b7bff577cd8","Type":"ContainerDied","Data":"770422b229afb7b4a9641d381acdd0581803b123dd80bf826047cf2fb2f1f478"}
Jul 01 19:53:55.347426 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.347349    2566 scope.go:117] "RemoveContainer" containerID="5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef"
Jul 01 19:53:55.356940 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.356916    2566 scope.go:117] "RemoveContainer" containerID="7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc"
Jul 01 19:53:55.364630 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.364609    2566 scope.go:117] "RemoveContainer" containerID="5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef"
Jul 01 19:53:55.364889 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:53:55.364871    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef\": container with ID starting with 5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef not found: ID does not exist" containerID="5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef"
Jul 01 19:53:55.364962 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.364902    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef"} err="failed to get container status \"5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef\": rpc error: code = NotFound desc = could not find container \"5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef\": container with ID starting with 5ddbd0129b6a74e9da2af6ee914e5c4d0417991dc45c244e7e840a809963c4ef not found: ID does not exist"
Jul 01 19:53:55.364962 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.364926    2566 scope.go:117] "RemoveContainer" containerID="7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc"
Jul 01 19:53:55.365209 ip-10-0-129-40 kubenswrapper[2566]: E0701 19:53:55.365190    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc\": container with ID starting with 7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc not found: ID does not exist" containerID="7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc"
Jul 01 19:53:55.365257 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.365216    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc"} err="failed to get container status \"7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc\": rpc error: code = NotFound desc = could not find container \"7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc\": container with ID starting with 7c3870ea46dea098ada6810c78b0b32d39a9cac93aa4b73ff393762c148ac0fc not found: ID does not exist"
Jul 01 19:53:55.368666 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.368646    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"]
Jul 01 19:53:55.372801 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.372782    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-74731-predictor-ddb44967c-gr2gj"]
Jul 01 19:53:55.918333 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:55.918300    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" path="/var/lib/kubelet/pods/c11a248b-5d39-4a65-9a5c-9b7bff577cd8/volumes"
Jul 01 19:53:58.302893 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:58.302852    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 01 19:53:58.344798 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:58.344772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:53:58.345231 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:53:58.345201    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 19:54:08.303600 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:08.303559    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 01 19:54:08.345460 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:08.345424    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 19:54:18.303587 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:18.303548    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 01 19:54:18.345594 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:18.345555    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 19:54:28.303249 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:28.303218    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 19:54:28.345739 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:28.345696    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 19:54:38.346298 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:54:38.346266    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 19:58:23.985730 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:58:23.985695    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 19:58:23.988279 ip-10-0-129-40 kubenswrapper[2566]: I0701 19:58:23.987196    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 20:03:06.430748 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:06.430707    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"]
Jul 01 20:03:06.431229 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:06.431027    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" containerID="cri-o://c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc" gracePeriod=30
Jul 01 20:03:06.431229 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:06.431030    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kube-rbac-proxy" containerID="cri-o://beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5" gracePeriod=30
Jul 01 20:03:07.262709 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:07.262675    2566 generic.go:356] "Generic (PLEG): container finished" podID="0e0a9972-6658-46fe-b0ce-20604af45554" containerID="beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5" exitCode=2
Jul 01 20:03:07.262886 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:07.262760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" event={"ID":"0e0a9972-6658-46fe-b0ce-20604af45554","Type":"ContainerDied","Data":"beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5"}
Jul 01 20:03:08.340418 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:08.340352    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 20:03:08.345746 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:08.345722    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 20:03:09.680002 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.679975    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 20:03:09.803651 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.803624    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e0a9972-6658-46fe-b0ce-20604af45554-proxy-tls\") pod \"0e0a9972-6658-46fe-b0ce-20604af45554\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") "
Jul 01 20:03:09.803845 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.803677    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9lmh\" (UniqueName: \"kubernetes.io/projected/0e0a9972-6658-46fe-b0ce-20604af45554-kube-api-access-p9lmh\") pod \"0e0a9972-6658-46fe-b0ce-20604af45554\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") "
Jul 01 20:03:09.803845 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.803712    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0e0a9972-6658-46fe-b0ce-20604af45554-kube-rbac-proxy-sar-config\") pod \"0e0a9972-6658-46fe-b0ce-20604af45554\" (UID: \"0e0a9972-6658-46fe-b0ce-20604af45554\") "
Jul 01 20:03:09.804102 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.804075    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e0a9972-6658-46fe-b0ce-20604af45554-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0e0a9972-6658-46fe-b0ce-20604af45554" (UID: "0e0a9972-6658-46fe-b0ce-20604af45554"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:03:09.805961 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.805932    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e0a9972-6658-46fe-b0ce-20604af45554-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0e0a9972-6658-46fe-b0ce-20604af45554" (UID: "0e0a9972-6658-46fe-b0ce-20604af45554"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:03:09.806058 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.806015    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0a9972-6658-46fe-b0ce-20604af45554-kube-api-access-p9lmh" (OuterVolumeSpecName: "kube-api-access-p9lmh") pod "0e0a9972-6658-46fe-b0ce-20604af45554" (UID: "0e0a9972-6658-46fe-b0ce-20604af45554"). InnerVolumeSpecName "kube-api-access-p9lmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 20:03:09.904390 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.904322    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e0a9972-6658-46fe-b0ce-20604af45554-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 20:03:09.904390 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.904386    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9lmh\" (UniqueName: \"kubernetes.io/projected/0e0a9972-6658-46fe-b0ce-20604af45554-kube-api-access-p9lmh\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 20:03:09.904390 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:09.904397    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0e0a9972-6658-46fe-b0ce-20604af45554-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 20:03:10.274817 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.274726    2566 generic.go:356] "Generic (PLEG): container finished" podID="0e0a9972-6658-46fe-b0ce-20604af45554" containerID="c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc" exitCode=0
Jul 01 20:03:10.274817 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.274771    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"
Jul 01 20:03:10.274994 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.274818    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" event={"ID":"0e0a9972-6658-46fe-b0ce-20604af45554","Type":"ContainerDied","Data":"c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc"}
Jul 01 20:03:10.274994 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.274879    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg" event={"ID":"0e0a9972-6658-46fe-b0ce-20604af45554","Type":"ContainerDied","Data":"9b3b02f330a95dfa3dbea6821074f735678f5ce0be4eb6594e9076c659378be7"}
Jul 01 20:03:10.274994 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.274910    2566 scope.go:117] "RemoveContainer" containerID="beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5"
Jul 01 20:03:10.283396 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.283351    2566 scope.go:117] "RemoveContainer" containerID="c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc"
Jul 01 20:03:10.290247 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.290231    2566 scope.go:117] "RemoveContainer" containerID="beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5"
Jul 01 20:03:10.290536 ip-10-0-129-40 kubenswrapper[2566]: E0701 20:03:10.290513    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5\": container with ID starting with beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5 not found: ID does not exist" containerID="beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5"
Jul 01 20:03:10.290616 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.290542    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5"} err="failed to get container status \"beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5\": rpc error: code = NotFound desc = could not find container \"beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5\": container with ID starting with beefabc94b3dae49d72ff84d984111e177e81280b0e955fa499863746a6db4b5 not found: ID does not exist"
Jul 01 20:03:10.290616 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.290559    2566 scope.go:117] "RemoveContainer" containerID="c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc"
Jul 01 20:03:10.290800 ip-10-0-129-40 kubenswrapper[2566]: E0701 20:03:10.290786    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc\": container with ID starting with c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc not found: ID does not exist" containerID="c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc"
Jul 01 20:03:10.290841 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.290802    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc"} err="failed to get container status \"c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc\": rpc error: code = NotFound desc = could not find container \"c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc\": container with ID starting with c15db52cb508093dace23c871f6215a1d601b99b02ca932a060613457bf38fbc not found: ID does not exist"
Jul 01 20:03:10.293673 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.293653    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"]
Jul 01 20:03:10.297683 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:10.297665    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b4182-predictor-58995c77cf-8brpg"]
Jul 01 20:03:11.917176 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:11.917141    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" path="/var/lib/kubelet/pods/0e0a9972-6658-46fe-b0ce-20604af45554/volumes"
Jul 01 20:03:24.008700 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:24.008592    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 20:03:24.013757 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:03:24.010638    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 20:08:24.027360 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:08:24.027256    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 20:08:24.031473 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:08:24.029617    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 20:11:00.786682 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:00.786646    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"]
Jul 01 20:11:00.787250 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:00.786944    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container" containerID="cri-o://3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482" gracePeriod=30
Jul 01 20:11:00.787250 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:00.787038    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kube-rbac-proxy" containerID="cri-o://187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e" gracePeriod=30
Jul 01 20:11:01.881395 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:01.881332    2566 generic.go:356] "Generic (PLEG): container finished" podID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerID="187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e" exitCode=2
Jul 01 20:11:01.881769 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:01.881413    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" event={"ID":"07cd3f38-e097-4fe2-b446-f6875dbd0ab9","Type":"ContainerDied","Data":"187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e"}
Jul 01 20:11:03.298662 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.298618    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 01 20:11:03.736427 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.736404    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 20:11:03.791000 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.790968    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-rbac-proxy-sar-config\") pod \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") "
Jul 01 20:11:03.791177 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.791019    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-proxy-tls\") pod \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") "
Jul 01 20:11:03.791177 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.791041    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2bcq\" (UniqueName: \"kubernetes.io/projected/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-api-access-q2bcq\") pod \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\" (UID: \"07cd3f38-e097-4fe2-b446-f6875dbd0ab9\") "
Jul 01 20:11:03.791415 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.791355    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "07cd3f38-e097-4fe2-b446-f6875dbd0ab9" (UID: "07cd3f38-e097-4fe2-b446-f6875dbd0ab9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:11:03.793163 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.793141    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-api-access-q2bcq" (OuterVolumeSpecName: "kube-api-access-q2bcq") pod "07cd3f38-e097-4fe2-b446-f6875dbd0ab9" (UID: "07cd3f38-e097-4fe2-b446-f6875dbd0ab9"). InnerVolumeSpecName "kube-api-access-q2bcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 20:11:03.793233 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.793166    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "07cd3f38-e097-4fe2-b446-f6875dbd0ab9" (UID: "07cd3f38-e097-4fe2-b446-f6875dbd0ab9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:11:03.890051 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.889959    2566 generic.go:356] "Generic (PLEG): container finished" podID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerID="3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482" exitCode=0
Jul 01 20:11:03.890051 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.890018    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"
Jul 01 20:11:03.890223 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.890047    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" event={"ID":"07cd3f38-e097-4fe2-b446-f6875dbd0ab9","Type":"ContainerDied","Data":"3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482"}
Jul 01 20:11:03.890223 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.890111    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95" event={"ID":"07cd3f38-e097-4fe2-b446-f6875dbd0ab9","Type":"ContainerDied","Data":"b861ed400baef7160200d708a56e62369c9f05aa37051570bb29940a364b2c93"}
Jul 01 20:11:03.890223 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.890137    2566 scope.go:117] "RemoveContainer" containerID="187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e"
Jul 01 20:11:03.892044 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.892023    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 20:11:03.892121 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.892052    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-proxy-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 20:11:03.892121 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.892068    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2bcq\" (UniqueName: \"kubernetes.io/projected/07cd3f38-e097-4fe2-b446-f6875dbd0ab9-kube-api-access-q2bcq\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\""
Jul 01 20:11:03.898264 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.898248    2566 scope.go:117] "RemoveContainer" containerID="3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482"
Jul 01 20:11:03.905817 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.905800    2566 scope.go:117] "RemoveContainer" containerID="187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e"
Jul 01 20:11:03.906066 ip-10-0-129-40 kubenswrapper[2566]: E0701 20:11:03.906048    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e\": container with ID starting with 187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e not found: ID does not exist" containerID="187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e"
Jul 01 20:11:03.906113 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.906075    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e"} err="failed to get container status \"187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e\": rpc error: code = NotFound desc = could not find container \"187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e\": container with ID starting with 187600907a730128a6ac43ea158a126c2994a16e376cd2a258dadcdc3aeace2e not found: ID does not exist"
Jul 01 20:11:03.906113 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.906092    2566 scope.go:117] "RemoveContainer" containerID="3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482"
Jul 01 20:11:03.906318 ip-10-0-129-40 kubenswrapper[2566]: E0701 20:11:03.906300    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482\": container with ID starting with 3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482 not found: ID does not exist" containerID="3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482"
Jul 01 20:11:03.906392 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.906328    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482"} err="failed to get container status \"3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482\": rpc error: code = NotFound desc = could not find container \"3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482\": container with ID starting with 3062168ac42d62226b1e780e22fcf55dd9fa6e770402f894a4b35148beff5482 not found: ID does not exist"
Jul 01 20:11:03.910319 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.910295    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"]
Jul 01 20:11:03.921253 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:03.921228    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ee34f-predictor-6bdb89f555-k5d95"]
Jul 01 20:11:05.917665 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:05.917632    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" path="/var/lib/kubelet/pods/07cd3f38-e097-4fe2-b446-f6875dbd0ab9/volumes"
Jul 01 20:11:32.053955 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.053920    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cn2r8/must-gather-dbd77"]
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054176    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054190    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054198    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054204    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054211    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054217    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054224    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054231    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054238    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054242    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054253    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054257    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054304    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054312    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054317    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0a9972-6658-46fe-b0ce-20604af45554" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054324    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c11a248b-5d39-4a65-9a5c-9b7bff577cd8" containerName="kube-rbac-proxy"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054329    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kserve-container"
Jul 01 20:11:32.054456 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.054334    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="07cd3f38-e097-4fe2-b446-f6875dbd0ab9" containerName="kube-rbac-proxy"
Jul 01 20:11:32.056962 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.056945    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.059436 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.059415    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cn2r8\"/\"openshift-service-ca.crt\""
Jul 01 20:11:32.060249 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.060231    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cn2r8\"/\"kube-root-ca.crt\""
Jul 01 20:11:32.060297 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.060267    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cn2r8\"/\"default-dockercfg-f6k5p\""
Jul 01 20:11:32.065548 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.065523    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cn2r8/must-gather-dbd77"]
Jul 01 20:11:32.084810 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.084778    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dbd2bbfa-fb14-459e-b705-8386c93f49d3-must-gather-output\") pod \"must-gather-dbd77\" (UID: \"dbd2bbfa-fb14-459e-b705-8386c93f49d3\") " pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.084810 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.084814    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77gfr\" (UniqueName: \"kubernetes.io/projected/dbd2bbfa-fb14-459e-b705-8386c93f49d3-kube-api-access-77gfr\") pod \"must-gather-dbd77\" (UID: \"dbd2bbfa-fb14-459e-b705-8386c93f49d3\") " pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.185872 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.185834    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dbd2bbfa-fb14-459e-b705-8386c93f49d3-must-gather-output\") pod \"must-gather-dbd77\" (UID: \"dbd2bbfa-fb14-459e-b705-8386c93f49d3\") " pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.185872 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.185870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77gfr\" (UniqueName: \"kubernetes.io/projected/dbd2bbfa-fb14-459e-b705-8386c93f49d3-kube-api-access-77gfr\") pod \"must-gather-dbd77\" (UID: \"dbd2bbfa-fb14-459e-b705-8386c93f49d3\") " pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.186192 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.186170    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dbd2bbfa-fb14-459e-b705-8386c93f49d3-must-gather-output\") pod \"must-gather-dbd77\" (UID: \"dbd2bbfa-fb14-459e-b705-8386c93f49d3\") " pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.194038 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.194005    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77gfr\" (UniqueName: \"kubernetes.io/projected/dbd2bbfa-fb14-459e-b705-8386c93f49d3-kube-api-access-77gfr\") pod \"must-gather-dbd77\" (UID: \"dbd2bbfa-fb14-459e-b705-8386c93f49d3\") " pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.366606 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.366521    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cn2r8/must-gather-dbd77"
Jul 01 20:11:32.497512 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.497463    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cn2r8/must-gather-dbd77"]
Jul 01 20:11:32.501343 ip-10-0-129-40 kubenswrapper[2566]: W0701 20:11:32.501314    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbd2bbfa_fb14_459e_b705_8386c93f49d3.slice/crio-cfca7ca6a7309dae530bc3e1332431ca3f6d410e9143471d07e7bc9306ea48d0 WatchSource:0}: Error finding container cfca7ca6a7309dae530bc3e1332431ca3f6d410e9143471d07e7bc9306ea48d0: Status 404 returned error can't find the container with id cfca7ca6a7309dae530bc3e1332431ca3f6d410e9143471d07e7bc9306ea48d0
Jul 01 20:11:32.503157 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.503142    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 20:11:32.986787 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:32.986748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cn2r8/must-gather-dbd77" event={"ID":"dbd2bbfa-fb14-459e-b705-8386c93f49d3","Type":"ContainerStarted","Data":"cfca7ca6a7309dae530bc3e1332431ca3f6d410e9143471d07e7bc9306ea48d0"}
Jul 01 20:11:33.993281 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:33.993190    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cn2r8/must-gather-dbd77" event={"ID":"dbd2bbfa-fb14-459e-b705-8386c93f49d3","Type":"ContainerStarted","Data":"118a921a57853f96215637e8285b0b71885526ba3535cdf7287a865ea8b0e959"}
Jul 01 20:11:33.993281 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:33.993245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cn2r8/must-gather-dbd77" event={"ID":"dbd2bbfa-fb14-459e-b705-8386c93f49d3","Type":"ContainerStarted","Data":"28660e982c06873ab32cdcef53d8735709bf01566089fc498cb2e5ca7ce1ce46"}
Jul 01 20:11:34.010449 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:34.010387    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cn2r8/must-gather-dbd77" podStartSLOduration=1.298987754 podStartE2EDuration="2.010350255s" totalImagesPullingTime="711.362501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:11:34.009179379 +0000 UTC m=+2890.558025758" watchObservedRunningTime="2026-07-01 20:11:34.010350255 +0000 UTC m=+2890.559196631"
Jul 01 20:11:34.623970 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:34.623931    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4wtck_ac905aae-0eb4-4ad7-9d9c-70da0d4d1b2d/global-pull-secret-syncer/0.log"
Jul 01 20:11:34.821633 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:34.821576    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zzn2q_ac5527dc-647a-4265-a9a9-d393f325753d/konnectivity-agent/0.log"
Jul 01 20:11:34.843185 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:34.843141    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-40.ec2.internal_851c30434749251be19b556dd27c3891/haproxy/0.log"
Jul 01 20:11:39.349422 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:39.349393    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pjdmz_be1b3505-bca3-4fda-8e84-7c883319a4c1/node-exporter/0.log"
Jul 01 20:11:39.377488 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:39.377447    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pjdmz_be1b3505-bca3-4fda-8e84-7c883319a4c1/kube-rbac-proxy/0.log"
Jul 01 20:11:39.402176 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:39.402146    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pjdmz_be1b3505-bca3-4fda-8e84-7c883319a4c1/init-textfile/0.log"
Jul 01 20:11:43.296514 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.296475    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"]
Jul 01 20:11:43.301342 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.301317    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.311228 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.311193    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"]
Jul 01 20:11:43.390939 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.390902    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-podres\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.391227 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.391200    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-proc\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.391326 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.391253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-lib-modules\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.391397 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.391330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-sys\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.391445 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.391425    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zjlw\" (UniqueName: \"kubernetes.io/projected/f48d1504-eb5d-459b-a063-734204983905-kube-api-access-6zjlw\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.492737 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492693    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zjlw\" (UniqueName: \"kubernetes.io/projected/f48d1504-eb5d-459b-a063-734204983905-kube-api-access-6zjlw\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.492924 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492759    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-podres\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.492924 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492790    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-proc\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.492924 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-lib-modules\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.492924 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-sys\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.492924 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492909    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-proc\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.493193 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492947    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-sys\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.493193 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.492968    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-podres\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.493193 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.493025    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f48d1504-eb5d-459b-a063-734204983905-lib-modules\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.503587 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.503560    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zjlw\" (UniqueName: \"kubernetes.io/projected/f48d1504-eb5d-459b-a063-734204983905-kube-api-access-6zjlw\") pod \"perf-node-gather-daemonset-79tkv\" (UID: \"f48d1504-eb5d-459b-a063-734204983905\") " pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.616515 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.616424    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:43.790851 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:43.790813    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"]
Jul 01 20:11:43.794683 ip-10-0-129-40 kubenswrapper[2566]: W0701 20:11:43.794655    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf48d1504_eb5d_459b_a063_734204983905.slice/crio-b141d77f5031c002d73a28d5cbd0adc17fab0d91f0dfd633612f7dec475eacbd WatchSource:0}: Error finding container b141d77f5031c002d73a28d5cbd0adc17fab0d91f0dfd633612f7dec475eacbd: Status 404 returned error can't find the container with id b141d77f5031c002d73a28d5cbd0adc17fab0d91f0dfd633612f7dec475eacbd
Jul 01 20:11:44.039336 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.039304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv" event={"ID":"f48d1504-eb5d-459b-a063-734204983905","Type":"ContainerStarted","Data":"6c5bafc3ec9c97931d4e5167eea230f7294256de1ffeff8106d594c37c03debd"}
Jul 01 20:11:44.039523 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.039354    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv" event={"ID":"f48d1504-eb5d-459b-a063-734204983905","Type":"ContainerStarted","Data":"b141d77f5031c002d73a28d5cbd0adc17fab0d91f0dfd633612f7dec475eacbd"}
Jul 01 20:11:44.039523 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.039472    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:44.056556 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.056505    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv" podStartSLOduration=1.056484285 podStartE2EDuration="1.056484285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:11:44.054997395 +0000 UTC m=+2900.603843770" watchObservedRunningTime="2026-07-01 20:11:44.056484285 +0000 UTC m=+2900.605330660"
Jul 01 20:11:44.066031 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.066003    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7bpz_998ccf1d-3c43-480e-8433-b4cb2cc2123a/dns/0.log"
Jul 01 20:11:44.090485 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.090454    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7bpz_998ccf1d-3c43-480e-8433-b4cb2cc2123a/kube-rbac-proxy/0.log"
Jul 01 20:11:44.194582 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.194503    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dkmcn_0e22d576-1039-4788-a13d-a98075ea84b4/dns-node-resolver/0.log"
Jul 01 20:11:44.707803 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.707759    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7dcb499546-hckkd_21836a16-868d-46fb-93e8-5da6f6b471f7/registry/0.log"
Jul 01 20:11:44.784882 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:44.784851    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zxrk2_193f723e-55a0-49d9-9f96-d222090fb79d/node-ca/0.log"
Jul 01 20:11:45.998741 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:45.998694    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-646vp_3bf8d620-9462-42bb-a12b-8204bd79abab/serve-healthcheck-canary/0.log"
Jul 01 20:11:46.437724 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:46.437698    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hwgxh_43fa70b6-35bc-49e9-83ba-44aed748412c/kube-rbac-proxy/0.log"
Jul 01 20:11:46.458845 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:46.458819    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hwgxh_43fa70b6-35bc-49e9-83ba-44aed748412c/exporter/0.log"
Jul 01 20:11:46.481283 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:46.481257    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hwgxh_43fa70b6-35bc-49e9-83ba-44aed748412c/extractor/0.log"
Jul 01 20:11:48.805795 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:48.805762    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-tfnfs_967fe994-d440-44d4-bfb0-cac168643fc2/server/0.log"
Jul 01 20:11:50.058071 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:50.058042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cn2r8/perf-node-gather-daemonset-79tkv"
Jul 01 20:11:54.466838 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:54.466801    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9njfs_7f93fdf9-24f9-44ff-8e51-583b8d5b8c30/kube-multus/0.log"
Jul 01 20:11:54.928899 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:54.928822    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/kube-multus-additional-cni-plugins/0.log"
Jul 01 20:11:54.955973 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:54.955947    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/egress-router-binary-copy/0.log"
Jul 01 20:11:54.980591 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:54.980561    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/cni-plugins/0.log"
Jul 01 20:11:55.004136 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:55.004108    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/bond-cni-plugin/0.log"
Jul 01 20:11:55.029468 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:55.029437    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/routeoverride-cni/0.log"
Jul 01 20:11:55.053810 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:55.053781    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/whereabouts-cni-bincopy/0.log"
Jul 01 20:11:55.075725 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:55.075697    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmrdr_2a2b3861-694a-4821-bda3-2d6f27e7567a/whereabouts-cni/0.log"
Jul 01 20:11:55.233125 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:55.233053    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qw5hn_aed27bed-2de0-4745-8ff1-b0c45f6a90ef/network-metrics-daemon/0.log"
Jul 01 20:11:55.253700 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:55.253670    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qw5hn_aed27bed-2de0-4745-8ff1-b0c45f6a90ef/kube-rbac-proxy/0.log"
Jul 01 20:11:56.460850 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.460822    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-controller/0.log"
Jul 01 20:11:56.480154 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.480119    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/1.log"
Jul 01 20:11:56.480309 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.480237    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovn-acl-logging/0.log"
Jul 01 20:11:56.500333 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.500308    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/kube-rbac-proxy-node/0.log"
Jul 01 20:11:56.522352 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.522309    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/kube-rbac-proxy-ovn-metrics/0.log"
Jul 01 20:11:56.543883 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.543852    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/northd/0.log"
Jul 01 20:11:56.567579 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.567551    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/nbdb/0.log"
Jul 01 20:11:56.592809 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.592769    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/sbdb/0.log"
Jul 01 20:11:56.634666 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:56.634638    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcgrj_8dc0ae12-7f88-49fe-8193-2c103f614edb/ovnkube-controller/0.log"
Jul 01 20:11:57.700251 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:57.700226    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-bdsqr_017149d2-7907-4bb3-8a16-780616d86470/network-check-target-container/0.log"
Jul 01 20:11:58.695725 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:58.695689    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-hnm9j_246949f7-3788-4e6f-8561-67b3ed764b7d/iptables-alerter/0.log"
Jul 01 20:11:59.399541 ip-10-0-129-40 kubenswrapper[2566]: I0701 20:11:59.399506    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-rxhp8_62099bca-1e18-41da-b8c0-968bde285901/tuned/0.log"