Jul 08 06:55:24.092887 ip-10-0-142-155 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.526051    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531224    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531239    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531243    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531247    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531250    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531253    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531256    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:55:24.578860 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531259    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531262    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531265    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531268    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531271    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531274    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531277    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531280    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531283    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531285    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531288    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531291    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531293    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531297    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531300    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531303    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531306    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531309    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531316    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531319    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:55:24.586448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531322    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531324    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531328    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531330    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531333    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531336    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531339    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531341    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531344    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531347    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531350    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531352    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531356    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531358    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531361    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531364    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531367    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531370    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531373    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531376    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:55:24.587134 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531378    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531381    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531384    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531387    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531391    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531394    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531396    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531399    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531402    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531405    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531407    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531410    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531413    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531415    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531420    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531423    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531426    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531429    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531431    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531434    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:55:24.587825 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531438    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531441    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531446    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531450    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531453    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531456    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531459    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531462    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531464    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531467    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531471    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531475    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531478    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531480    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531483    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531486    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531489    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531491    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531494    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:55:24.588508 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531497    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531500    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531503    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531507    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531510    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531512    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531517    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531522    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531527    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531530    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531534    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531536    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.531539    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532353    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532360    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532363    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532366    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532369    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532372    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:55:24.589229 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532375    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532378    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532381    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532383    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532386    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532391    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532394    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532397    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532400    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532403    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532406    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532408    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532411    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532414    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532416    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532419    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532422    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532424    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532427    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532430    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:55:24.589915 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532433    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532435    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532438    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532441    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532444    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532447    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532450    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532452    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532455    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532458    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532461    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532464    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532467    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532469    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532472    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532475    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532478    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532481    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532485    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532488    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:55:24.590603 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532491    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532493    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532496    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532498    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532501    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532504    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532506    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532510    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532515    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532519    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532522    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532525    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532528    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532532    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532535    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532538    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532541    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532544    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532546    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:55:24.591284 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532549    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532552    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532555    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532558    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532561    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532563    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532566    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532569    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532571    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532575    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532579    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532583    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532587    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532590    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532592    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532595    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532598    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532600    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532603    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:55:24.591982 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532608    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532611    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532614    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532617    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532619    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532622    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532625    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532628    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532631    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532649    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532653    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532658    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532661    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532664    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.532666    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533074    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533080    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533083    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533087    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533089    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:55:24.592687 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533092    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533095    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533098    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533101    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533104    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533107    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533110    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533114    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533117    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533119    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533122    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533125    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533146    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533150    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533153    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533156    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533159    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533161    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533164    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533167    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:55:24.593374 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533170    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533173    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533176    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533180    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533183    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533186    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533188    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533193    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533199    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533204    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533208    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533212    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533217    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533221    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533226    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533230    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533233    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533235    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533239    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:55:24.594061 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533242    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533245    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533249    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533251    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533254    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533257    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533260    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533263    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533266    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533269    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533271    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533274    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533278    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533282    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533286    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533289    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533292    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533295    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533298    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533301    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:55:24.594972 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533304    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533306    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533312    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533315    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533317    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533320    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533323    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533326    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533328    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533331    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533333    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533336    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533340    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533343    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533346    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533348    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533352    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533355    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533358    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533360    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:55:24.595670 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533364    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533366    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533369    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533372    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533374    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533377    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533380    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533383    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533385    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533388    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533391    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533393    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533399    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533402    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.533406    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533485    2585 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533496    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533505    2585 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533514    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533520    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533525    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 06:55:24.596342 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533532    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533539    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533542    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533546    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533550    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533554    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533557    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533561    2585 flags.go:64] FLAG: --cgroup-root=""
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533564    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533567    2585 flags.go:64] FLAG: --client-ca-file=""
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533570    2585 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533573    2585 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533579    2585 flags.go:64] FLAG: --cluster-domain=""
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533582    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533585    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533588    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533592    2585 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533596    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533599    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533602    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533606    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533609    2585 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533612    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533618    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 06:55:24.597021 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533621    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533626    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533630    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533648    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533652    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533655    2585 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533658    2585 flags.go:64] FLAG: --enable-server="true"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533661    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533666    2585 flags.go:64] FLAG: --event-burst="100"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533670    2585 flags.go:64] FLAG: --event-qps="50"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533673    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533676    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533679    2585 flags.go:64] FLAG: --eviction-hard=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533684    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533687    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533691    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533694    2585 flags.go:64] FLAG: --eviction-soft=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533697    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533701    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533704    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533707    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533710    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533713    2585 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533716    2585 flags.go:64] FLAG: --feature-gates=""
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533720    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 06:55:24.597709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533724    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533727    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533730    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533733    2585 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533737    2585 flags.go:64] FLAG: --help="false"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533740    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-142-155.ec2.internal"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533745    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533748    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533752    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533756    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533760    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533763    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533766    2585 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533769    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533772    2585 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533775    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533779    2585 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533782    2585 flags.go:64] FLAG: --kube-reserved=""
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533785    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533788    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533791    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533795    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533798    2585 flags.go:64] FLAG: --lock-file=""
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533801    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 06:55:24.598625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533804    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533808    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533813    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533816    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533819    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533822    2585 flags.go:64] FLAG: --logging-format="text"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533825    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533829    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533832    2585 flags.go:64] FLAG: --manifest-url=""
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533835    2585 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533840    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533842    2585 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533853    2585 flags.go:64] FLAG: --max-pods="110"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533856    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533861    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533864    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533869    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533872    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533875    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533878    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533887    2585 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533890    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533893    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 06:55:24.662892 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533896    2585 flags.go:64] FLAG: --pod-cidr=""
Jul 08 06:55:24.641565 ip-10-0-142-155 systemd[1]: Started Kubernetes Kubelet.
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533900    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533906    2585 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533909    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533913    2585 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533916    2585 flags.go:64] FLAG: --port="10250"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533919    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533923    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-062ee3baded289a39"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533926    2585 flags.go:64] FLAG: --qos-reserved=""
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533930    2585 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533933    2585 flags.go:64] FLAG: --register-node="true"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533936    2585 flags.go:64] FLAG: --register-with-taints=""
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533940    2585 flags.go:64] FLAG: --registry-burst="10"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533943    2585 flags.go:64] FLAG: --registry-qps="5"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533946    2585 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533949    2585 flags.go:64] FLAG: --reserved-memory=""
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533953    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533956    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533960    2585 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533963    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533966    2585 flags.go:64] FLAG: --runonce="false"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533969    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533972    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533975    2585 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533980    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 06:55:24.664188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533984    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533987    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533991    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533994    2585 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.533997    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534000    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534003    2585 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534006    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534010    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534013    2585 flags.go:64] FLAG: --system-cgroups=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534016    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534022    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534025    2585 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534028    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534032    2585 flags.go:64] FLAG: --tls-min-version=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534036    2585 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534039    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534042    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534045    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534048    2585 flags.go:64] FLAG: --v="2"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534053    2585 flags.go:64] FLAG: --version="false"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534058    2585 flags.go:64] FLAG: --vmodule=""
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534062    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.534065    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534193    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:55:24.668072 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534197    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534200    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534204    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534207    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534210    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534213    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534217    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534221    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534224    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534227    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534232    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534236    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534239    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534242    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534245    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534249    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534253    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534256    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534259    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:55:24.669771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534262    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534265    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534268    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534271    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534274    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534278    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534281    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534284    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534287    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534289    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534292    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534295    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534298    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534301    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534304    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534306    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534309    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534312    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534316    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534322    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:55:24.672128 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534326    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534329    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534332    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534335    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534338    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534341    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534344    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534347    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534349    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534352    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534355    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534357    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534360    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534363    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534366    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534368    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534371    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534374    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534377    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534380    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:55:24.673281 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534382    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534385    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534388    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534390    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534393    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534396    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534399    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534401    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534404    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534407    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534409    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534413    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534418    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534421    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534424    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534427    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534429    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534432    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534435    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:55:24.674220 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534437    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534440    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534443    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534446    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534448    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534451    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534454    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534456    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534459    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534462    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534464    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534467    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534470    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534473    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534475    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534478    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534481    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534483    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534486    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:55:24.751096 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.534489    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.535089    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.542651    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.542669    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.542686    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.542692    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542755    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542761    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542765    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542768    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542773    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542777    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542781    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542784    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542788    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:55:24.751947 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542791    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542794    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542797    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542800    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542803    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542807    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542809    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542813    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542817    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542820    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542824    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542827    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542830    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542833    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542836    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542839    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542842    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542845    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542848    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542851    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:55:24.752695 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542855    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542858    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542861    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542863    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542866    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542869    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542871    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542874    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542877    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542880    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542883    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542886    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542888    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542891    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542894    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542897    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542903    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542906    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542909    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542912    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:55:24.753503 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542915    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542917    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542920    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542923    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542926    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542929    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542932    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542935    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542938    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542941    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542944    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542946    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542950    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542953    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542955    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542958    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542961    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542964    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542967    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542970    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:55:24.754367 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542973    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542975    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542978    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542981    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542983    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542986    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542989    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542992    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.542996    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543000    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543004    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543007    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543010    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543013    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543016    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543020    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543023    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543026    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543028    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:55:24.755265 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543031    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543034    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543037    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543040    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543044    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543047    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543050    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543054    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543056    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543059    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543062    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.543068    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543195    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543200    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543203    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:55:24.755958 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543205    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543208    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543211    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543213    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543216    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543220    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543226    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543229    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543232    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543235    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543237    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543240    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543243    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543246    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543249    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543251    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543254    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543257    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543260    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543263    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:55:24.756413 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543266    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543269    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543275    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543277    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543280    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543283    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543287    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543291    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543296    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543299    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543302    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543305    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543308    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543311    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543314    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543317    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543319    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543322    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543326    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543329    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:55:24.757056 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543331    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543334    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543337    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543340    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543342    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543345    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543348    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543350    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543353    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543355    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543358    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543361    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543363    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543367    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543372    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543375    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543378    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543380    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543384    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:55:24.757625 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543387    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543389    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543392    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543395    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543397    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543400    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543403    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543405    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543408    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543411    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543414    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543417    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543420    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543423    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543425    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543428    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543431    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543434    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543436    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543439    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:55:24.758219 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543442    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543444    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543447    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543451    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543453    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543456    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543459    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543464    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543468    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543471    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543474    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543477    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543480    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543483    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543486    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543489    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:55:24.758886 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:24.543491    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.543497    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.543614    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.546003    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.547606    2585 server.go:1019] "Starting client certificate rotation"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.547713    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.547784    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.566117    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.571022    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.600270    2585 log.go:25] "Validated CRI v1 runtime API"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.603027    2585 log.go:25] "Validated CRI v1 image API"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.605442    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 06:55:24.759393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.608257    2585 fs.go:135] Filesystem UUIDs: map[73fd3dbd-ad60-440f-a4aa-04e7ea50e50c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c9974ac1-96e1-4036-9ec8-c45829d5e2b9:/dev/nvme0n1p4]
Jul 08 06:55:24.759795 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.608279    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.615241    2585 manager.go:216] Machine: {Timestamp:2026-07-08 06:55:24.614441904 +0000 UTC m=+0.401278216 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099190 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2152be8f1e990aad7752d194324a8b SystemUUID:ec2152be-8f1e-990a-ad77-52d194324a8b BootID:d9f8aa9a-badd-43c4-9067-cc1abfb94dd1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:a3:d5:c2:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:a3:d5:c2:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:d1:1f:3a:36:4d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.615404    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.615523    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.619031    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.619072    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-155.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.619911    2585 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.619924    2585 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 06:55:24.759907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.619937    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.621995    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.623316    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.627789    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.628017    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629008    2585 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629030    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629052    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629066    2585 kubelet.go:398] "Adding apiserver pod source"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629080    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629537    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.629558    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.632268    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.634500    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.634544    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636464    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636482    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636488    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636495    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636504    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636511    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636521    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636527    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636535    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636541    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636550    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636556    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636580    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.636586    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.637603    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.640904    2585 server.go:1268] "Started kubelet"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.645162    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.645228    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.645266    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.652939    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.653143    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vjw8g"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.653274    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-155.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.654520    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 06:55:24.760203 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.655460    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.668853    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.669975    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.670079    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vjw8g"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.670218    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.670657    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.670674    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.670857    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.670927    2585 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.670934    2585 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.671218    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.674716    2585 factory.go:55] Registering systemd factory
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.674734    2585 factory.go:223] Registration of the systemd container factory successfully
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675002    2585 factory.go:153] Registering CRI-O factory
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675018    2585 factory.go:223] Registration of the crio container factory successfully
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675065    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675092    2585 factory.go:103] Registering Raw factory
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675108    2585 manager.go:1196] Started watching for new ooms in manager
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675364    2585 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.675712    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.675808    2585 manager.go:349] Starting recovery of all containers
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.675837    2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.675734    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-155.ec2.internal.18c03e17dc5baf85  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-155.ec2.internal,UID:ip-10-0-142-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-155.ec2.internal,},FirstTimestamp:2026-07-08 06:55:24.640882565 +0000 UTC m=+0.427718875,LastTimestamp:2026-07-08 06:55:24.640882565 +0000 UTC m=+0.427718875,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-155.ec2.internal,}"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.687182    2585 manager.go:354] Recovery completed
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.693508    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.697524    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.697558    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.697569    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.699060    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.699070    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.699071    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.699089    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.700014    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:24.761364 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.700040    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:24.762619 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.700050    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:24.762619 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.725423    2585 policy_none.go:49] "None policy: Start"
Jul 08 06:55:24.762619 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.725441    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 06:55:24.762619 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.725454    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 06:55:24.762619 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.727805    2585 policy_none.go:47] "Start"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.770815    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.797574    2585 manager.go:337] "Starting Device Plugin manager"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.797627    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.797640    2585 server.go:87] "Starting device plugin registration server"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.797941    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.797953    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.798059    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.798131    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.798140    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.798616    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 06:55:24.861584 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.798678    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:24.885510 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.882088    2585 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-155.ec2.internal\" not found" node="ip-10-0-142-155.ec2.internal"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.898060    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.899140    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.899173    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.899189    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.899220    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-155.ec2.internal"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.912126    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-155.ec2.internal"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.912150    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-155.ec2.internal\": node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.942687    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.944076    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.944116    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.944139    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.944175    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:24.951465    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:55:24.998862 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:24.954694    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.047070    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"]
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.047169    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.048298    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.048325    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.048335    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.051183    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.051360    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.051390    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.052042    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.052061    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.052070    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.052452    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.052470    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.052481    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.054520    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.054542    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.054723    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.055213    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.055231    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.055240    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.072610    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14c76a2688b1ab5dfa064b62e6726935-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal\" (UID: \"14c76a2688b1ab5dfa064b62e6726935\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.072634    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14c76a2688b1ab5dfa064b62e6726935-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal\" (UID: \"14c76a2688b1ab5dfa064b62e6726935\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.072652    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8aeda92cfb515398ae0d19a52ccec463-config\") pod \"kube-apiserver-proxy-ip-10-0-142-155.ec2.internal\" (UID: \"8aeda92cfb515398ae0d19a52ccec463\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.089134    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-155.ec2.internal\" not found" node="ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.123178 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.094263    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-155.ec2.internal\" not found" node="ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.155630    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.176521    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14c76a2688b1ab5dfa064b62e6726935-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal\" (UID: \"14c76a2688b1ab5dfa064b62e6726935\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.176553    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8aeda92cfb515398ae0d19a52ccec463-config\") pod \"kube-apiserver-proxy-ip-10-0-142-155.ec2.internal\" (UID: \"8aeda92cfb515398ae0d19a52ccec463\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.176630    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14c76a2688b1ab5dfa064b62e6726935-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal\" (UID: \"14c76a2688b1ab5dfa064b62e6726935\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.176643    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14c76a2688b1ab5dfa064b62e6726935-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal\" (UID: \"14c76a2688b1ab5dfa064b62e6726935\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.176657    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8aeda92cfb515398ae0d19a52ccec463-config\") pod \"kube-apiserver-proxy-ip-10-0-142-155.ec2.internal\" (UID: \"8aeda92cfb515398ae0d19a52ccec463\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.176681    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14c76a2688b1ab5dfa064b62e6726935-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal\" (UID: \"14c76a2688b1ab5dfa064b62e6726935\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.261969 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.256838    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.403401 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.359023    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.403401 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.392271    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.409218 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.409055    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:25.527077 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.459793    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.649550 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.546486    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 06:55:25.649550 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.546711    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 06:55:25.649550 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.560633    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.780509 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.660863    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.780509 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.674296    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 06:50:25 +0000 UTC" deadline="2027-12-05 15:27:15.680862138 +0000 UTC"
Jul 08 06:55:25.780509 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.674323    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12368h31m50.00654308s"
Jul 08 06:55:25.780509 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.766236    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.901327 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.791252    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:55:25.901327 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:25.869077    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-155.ec2.internal\" not found"
Jul 08 06:55:25.901327 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.895843    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:55:26.003776 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.913725    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:55:26.003776 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.970508    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:26.003776 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.996328    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 06:55:26.003776 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:25.997259    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal"
Jul 08 06:55:26.008705 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.008688    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 06:55:26.159280 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:26.159214    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c76a2688b1ab5dfa064b62e6726935.slice/crio-01aac124b13461789e642afac279a3f5cf00217ee437d7206c8f4a1d8ed8d853 WatchSource:0}: Error finding container 01aac124b13461789e642afac279a3f5cf00217ee437d7206c8f4a1d8ed8d853: Status 404 returned error can't find the container with id 01aac124b13461789e642afac279a3f5cf00217ee437d7206c8f4a1d8ed8d853
Jul 08 06:55:26.165335 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.165312    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 06:55:26.172881 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:26.172856    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aeda92cfb515398ae0d19a52ccec463.slice/crio-7320547f3d565f4cd097318f45eb693285d18fa5538c5ab2b0a8820fb1397fb0 WatchSource:0}: Error finding container 7320547f3d565f4cd097318f45eb693285d18fa5538c5ab2b0a8820fb1397fb0: Status 404 returned error can't find the container with id 7320547f3d565f4cd097318f45eb693285d18fa5538c5ab2b0a8820fb1397fb0
Jul 08 06:55:26.546803 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.546763    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:55:26.632699 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.632665    2585 apiserver.go:52] "Watching apiserver"
Jul 08 06:55:26.641257 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.641229    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 06:55:26.641724 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.641692    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-mlrcx","kube-system/konnectivity-agent-qpzwf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p","openshift-multus/multus-additional-cni-plugins-lsgx2","openshift-network-diagnostics/network-check-target-j48lr","openshift-network-operator/iptables-alerter-tc5xm","openshift-ovn-kubernetes/ovnkube-node-v4jqd","kube-system/global-pull-secret-syncer-xpq58","kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bb65s","openshift-dns/node-resolver-mczhb","openshift-image-registry/node-ca-k74tn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal","openshift-multus/multus-lkzph"]
Jul 08 06:55:26.644489 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.644465    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.646929 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.646910    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.647506 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.647485    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 06:55:26.648202 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.648046    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.648202 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.648077    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 06:55:26.648202 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.648115    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.648202 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.648129    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 06:55:26.648202 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.648150    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-54q54\""
Jul 08 06:55:26.649272 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.649246    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.649467 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.649446    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 06:55:26.651949 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.651904    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.652675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.652010    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.653075 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.652791    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.654477 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.654454    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:26.655474 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.655026    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:26.658507 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.657873    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vhf46\""
Jul 08 06:55:26.658507 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.657958    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 06:55:26.658507 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.658047    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.658507 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.658082    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.658507 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.658405    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.658821 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.658780    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 06:55:26.658884 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.658867    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xpvzr\""
Jul 08 06:55:26.658884 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.658873    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.659050 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.659029    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 06:55:26.659189 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.659171    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.659303 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.659281    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 06:55:26.659939 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.659920    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fhgnt\""
Jul 08 06:55:26.660767 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.660747    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.663415 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.663355    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.663530 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.663427    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:26.665968 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.665949    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.666220 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.666125    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.666350 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.666324    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 06:55:26.666809 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.666778    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 06:55:26.666992 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.666976    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t5tj7\""
Jul 08 06:55:26.667103 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.667062    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l9d42\""
Jul 08 06:55:26.667638 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.667622    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.669295 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.669184    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5xnn8\""
Jul 08 06:55:26.669484 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.669313    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 06:55:26.669484 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.669391    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 06:55:26.669626 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.669545    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 06:55:26.669738 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.669708    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.672187 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.672165    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:26.672299 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.672253    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:26.673667 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.673553    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.673781 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.673743    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.673847 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.673795    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4hglb\""
Jul 08 06:55:26.674645 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.674623    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.680116 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.680096    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 06:55:26.680242 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.680225    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 06:55:26.680337 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.680261    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 06:55:26.680337 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.680292    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-788q7\""
Jul 08 06:55:26.686139 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686065    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-modprobe-d\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.686139 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686103    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-lib-modules\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.686139 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686129    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-var-lib-kubelet\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.686366 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686153    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.686366 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686179    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.686366 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686204    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-systemd\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.686366 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686270    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.686366 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686349    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-cni-multus\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686392    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686443    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-hosts-file\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-device-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b79cf45f-e70a-412a-8b70-f1b8066a066d-iptables-alerter-script\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686591    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fcd81370-ee72-4e93-9089-44790495c67b-dbus\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686634    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-node-log\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.686772 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686670    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-cni-bin\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686721    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-ovnkube-config\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686841    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-ovnkube-script-lib\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686888    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-kubelet\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686915    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lgjc\" (UniqueName: \"kubernetes.io/projected/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-kube-api-access-9lgjc\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686941    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v68gz\" (UniqueName: \"kubernetes.io/projected/693c96e8-8277-46a4-8a67-216e9673cdba-kube-api-access-v68gz\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.686983    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b79cf45f-e70a-412a-8b70-f1b8066a066d-host-slash\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687008    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-os-release\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687032    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-system-cni-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687082    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-conf-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687118    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-systemd\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-system-cni-dir\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687183    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysconfig\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687198    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-sys-fs\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687212    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-os-release\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687264    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-k8s-cni-cncf-io\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687295    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-sys\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687320    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-cnibin\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687357    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-cni-bin\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687390    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91e000e0-f8fc-4045-84cb-ae1a8b8cae39-agent-certs\") pod \"konnectivity-agent-qpzwf\" (UID: \"91e000e0-f8fc-4045-84cb-ae1a8b8cae39\") " pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687431    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-tmp-dir\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687449    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-systemd-units\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687473    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5llnh\" (UniqueName: \"kubernetes.io/projected/eb5ab310-b866-4cda-86fc-689295b20301-kube-api-access-5llnh\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687496    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/679705c9-0c44-4084-9bae-366fe27be3e7-tmp\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.687529 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687523    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn82l\" (UniqueName: \"kubernetes.io/projected/6de506fd-0787-407c-97e3-349973298b4f-kube-api-access-jn82l\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687548    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687574    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-registration-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687620    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687640    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-env-overrides\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687683    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687712    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687745    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-ovn\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687811    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-daemon-config\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687843    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcrvp\" (UniqueName: \"kubernetes.io/projected/b79cf45f-e70a-412a-8b70-f1b8066a066d-kube-api-access-jcrvp\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687910    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-etc-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687929    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-multus-certs\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687954    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-socket-dir-parent\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.688166 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.687977    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-hostroot\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688006    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91e000e0-f8fc-4045-84cb-ae1a8b8cae39-konnectivity-ca\") pod \"konnectivity-agent-qpzwf\" (UID: \"91e000e0-f8fc-4045-84cb-ae1a8b8cae39\") " pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688032    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysctl-d\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688056    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-socket-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688093    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cnibin\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688122    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlksj\" (UniqueName: \"kubernetes.io/projected/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-kube-api-access-jlksj\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688167    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688218    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-netns\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688242    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysctl-conf\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688273    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fcd81370-ee72-4e93-9089-44790495c67b-kubelet-config\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688294    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-kubelet\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688320    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-var-lib-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688353    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.688778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688389    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-cni-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688411    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-kubernetes\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-host\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688461    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/679705c9-0c44-4084-9bae-366fe27be3e7-etc-tuned\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688485    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-slash\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688509    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-run-ovn-kubernetes\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688536    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-cni-netd\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688587    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-cni-binary-copy\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688646    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-run\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688690    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95pw7\" (UniqueName: \"kubernetes.io/projected/679705c9-0c44-4084-9bae-366fe27be3e7-kube-api-access-95pw7\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688714    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sck\" (UniqueName: \"kubernetes.io/projected/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-kube-api-access-k6sck\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688738    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-run-netns\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688761    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-log-socket\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688798    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb5ab310-b866-4cda-86fc-689295b20301-ovn-node-metrics-cert\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.689405 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688836    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cni-binary-copy\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.690060 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.688863    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-etc-kubernetes\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.693675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.693340    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 06:55:26.720369 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.720339    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpszg"
Jul 08 06:55:26.731029 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.731001    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpszg"
Jul 08 06:55:26.771752 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.771722    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789189    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-ovnkube-config\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789231    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-ovnkube-script-lib\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789264    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-host\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789290    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-serviceca\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789355    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-kubelet\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789379    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lgjc\" (UniqueName: \"kubernetes.io/projected/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-kube-api-access-9lgjc\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789405    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v68gz\" (UniqueName: \"kubernetes.io/projected/693c96e8-8277-46a4-8a67-216e9673cdba-kube-api-access-v68gz\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789431    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b79cf45f-e70a-412a-8b70-f1b8066a066d-host-slash\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.789458 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789455    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-os-release\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789479    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-system-cni-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789505    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-conf-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789529    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-systemd\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789554    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-system-cni-dir\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789588    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysconfig\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789634    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-sys-fs\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789659    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-os-release\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-k8s-cni-cncf-io\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789690    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-system-cni-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789711    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-sys\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789753    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b79cf45f-e70a-412a-8b70-f1b8066a066d-host-slash\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789782    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-sys\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789787    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-cnibin\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789821    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-cni-bin\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789835    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-cnibin\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789851    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91e000e0-f8fc-4045-84cb-ae1a8b8cae39-agent-certs\") pod \"konnectivity-agent-qpzwf\" (UID: \"91e000e0-f8fc-4045-84cb-ae1a8b8cae39\") " pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789907    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-ovnkube-config\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789936    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-sys-fs\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789957    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-os-release\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.789989    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-tmp-dir\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790010    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysconfig\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790014    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-systemd\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790034    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-system-cni-dir\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790056    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-systemd-units\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790064    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-os-release\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790080    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-k8s-cni-cncf-io\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790087    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5llnh\" (UniqueName: \"kubernetes.io/projected/eb5ab310-b866-4cda-86fc-689295b20301-kube-api-access-5llnh\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790115    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-conf-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790114    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/679705c9-0c44-4084-9bae-366fe27be3e7-tmp\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.790888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790211    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-systemd-units\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790217    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790319    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn82l\" (UniqueName: \"kubernetes.io/projected/6de506fd-0787-407c-97e3-349973298b4f-kube-api-access-jn82l\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790352    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790380    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-registration-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790451    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790481    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-env-overrides\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790507    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790535    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790563    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-ovn\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790508    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-tmp-dir\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790620    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb76t\" (UniqueName: \"kubernetes.io/projected/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-kube-api-access-lb76t\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790667    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-daemon-config\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790697    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcrvp\" (UniqueName: \"kubernetes.io/projected/b79cf45f-e70a-412a-8b70-f1b8066a066d-kube-api-access-jcrvp\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.791675 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790725    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-etc-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790751    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-multus-certs\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790766    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790799    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790848    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-multus-certs\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790865    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-registration-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790884    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-cni-bin\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.790906    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791002    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-socket-dir-parent\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791035    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-hostroot\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791060    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91e000e0-f8fc-4045-84cb-ae1a8b8cae39-konnectivity-ca\") pod \"konnectivity-agent-qpzwf\" (UID: \"91e000e0-f8fc-4045-84cb-ae1a8b8cae39\") " pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791211    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-kubelet\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791210    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791273    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-hostroot\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.792445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791272    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysctl-d\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791311    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-socket-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791328    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-env-overrides\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791341    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cnibin\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791383    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-etc-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791399    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysctl-d\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791423    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-ovn\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791450    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb5ab310-b866-4cda-86fc-689295b20301-ovnkube-script-lib\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791459    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cnibin\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791454    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-socket-dir-parent\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791488    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlksj\" (UniqueName: \"kubernetes.io/projected/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-kube-api-access-jlksj\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791538    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791549    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-socket-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791579    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-netns\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.793225 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791619    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-daemon-config\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791691    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysctl-conf\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791730    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-run-netns\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791791    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fcd81370-ee72-4e93-9089-44790495c67b-kubelet-config\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791854    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fcd81370-ee72-4e93-9089-44790495c67b-kubelet-config\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791880    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-sysctl-conf\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791895    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-kubelet\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791942    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-kubelet\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791955    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-var-lib-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791985    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91e000e0-f8fc-4045-84cb-ae1a8b8cae39-konnectivity-ca\") pod \"konnectivity-agent-qpzwf\" (UID: \"91e000e0-f8fc-4045-84cb-ae1a8b8cae39\") " pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.791990    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792021    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-cni-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792050    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-kubernetes\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792073    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-host\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792096    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/679705c9-0c44-4084-9bae-366fe27be3e7-etc-tuned\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.793983 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792103    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-var-lib-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792058    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-openvswitch\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792121    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-slash\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792144    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-kubernetes\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792148    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-run-ovn-kubernetes\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792180    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-run-ovn-kubernetes\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792188    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-cni-netd\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792195    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-slash\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792118    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-multus-cni-dir\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792156    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-host\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792216    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-cni-binary-copy\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792240    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-cni-netd\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792243    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-run\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792274    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95pw7\" (UniqueName: \"kubernetes.io/projected/679705c9-0c44-4084-9bae-366fe27be3e7-kube-api-access-95pw7\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792284    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-run\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.794703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792301    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sck\" (UniqueName: \"kubernetes.io/projected/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-kube-api-access-k6sck\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792410    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-run-netns\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792440    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-log-socket\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792448    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-run-netns\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792464    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb5ab310-b866-4cda-86fc-689295b20301-ovn-node-metrics-cert\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792493    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-log-socket\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792492    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cni-binary-copy\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792544    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-etc-kubernetes\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-modprobe-d\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792641    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-lib-modules\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792666    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-var-lib-kubelet\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792729    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-cni-binary-copy\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792737    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.795434 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792763    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-systemd\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792805    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792835    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-cni-multus\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792859    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-lib-modules\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792862    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792899    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-hosts-file\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792925    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-device-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.792934    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792952    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b79cf45f-e70a-412a-8b70-f1b8066a066d-iptables-alerter-script\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792967    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-etc-kubernetes\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.792978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fcd81370-ee72-4e93-9089-44790495c67b-dbus\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.793035    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:55:27.293003619 +0000 UTC m=+3.079839939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793041    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-device-dir\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793055    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-node-log\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.796020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793087    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-cni-bin\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793091    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-etc-modprobe-d\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793124    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-hosts-file\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793170    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-run-systemd\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793204    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/693c96e8-8277-46a4-8a67-216e9673cdba-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793243    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-node-log\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793287    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb5ab310-b866-4cda-86fc-689295b20301-host-cni-bin\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793088    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fcd81370-ee72-4e93-9089-44790495c67b-dbus\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793032    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-cni-binary-copy\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793330    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-host-var-lib-cni-multus\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793382    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/679705c9-0c44-4084-9bae-366fe27be3e7-var-lib-kubelet\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.793390    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.793450    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:55:27.293433673 +0000 UTC m=+3.080269981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793519    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b79cf45f-e70a-412a-8b70-f1b8066a066d-iptables-alerter-script\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.796842 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.793603    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.797495 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.794996    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/679705c9-0c44-4084-9bae-366fe27be3e7-etc-tuned\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.797495 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.795011    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/679705c9-0c44-4084-9bae-366fe27be3e7-tmp\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.797495 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.795682    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91e000e0-f8fc-4045-84cb-ae1a8b8cae39-agent-certs\") pod \"konnectivity-agent-qpzwf\" (UID: \"91e000e0-f8fc-4045-84cb-ae1a8b8cae39\") " pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:26.797495 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.796708    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb5ab310-b866-4cda-86fc-689295b20301-ovn-node-metrics-cert\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.802709 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.802688    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:26.802873 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.802718    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:26.802873 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.802732    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:26.802873 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:26.802811    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:55:27.302791054 +0000 UTC m=+3.089627369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:26.804020 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.803948    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lgjc\" (UniqueName: \"kubernetes.io/projected/8520e981-4ad3-4258-bd34-b9eb7bdcecd4-kube-api-access-9lgjc\") pod \"multus-lkzph\" (UID: \"8520e981-4ad3-4258-bd34-b9eb7bdcecd4\") " pod="openshift-multus/multus-lkzph"
Jul 08 06:55:26.807183 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.807104    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95pw7\" (UniqueName: \"kubernetes.io/projected/679705c9-0c44-4084-9bae-366fe27be3e7-kube-api-access-95pw7\") pod \"tuned-bb65s\" (UID: \"679705c9-0c44-4084-9bae-366fe27be3e7\") " pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:26.807357 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.807328    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn82l\" (UniqueName: \"kubernetes.io/projected/6de506fd-0787-407c-97e3-349973298b4f-kube-api-access-jn82l\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:26.807804 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.807778    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcrvp\" (UniqueName: \"kubernetes.io/projected/b79cf45f-e70a-412a-8b70-f1b8066a066d-kube-api-access-jcrvp\") pod \"iptables-alerter-tc5xm\" (UID: \"b79cf45f-e70a-412a-8b70-f1b8066a066d\") " pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.807902 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.807842    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5llnh\" (UniqueName: \"kubernetes.io/projected/eb5ab310-b866-4cda-86fc-689295b20301-kube-api-access-5llnh\") pod \"ovnkube-node-v4jqd\" (UID: \"eb5ab310-b866-4cda-86fc-689295b20301\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.808836 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.808788    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlksj\" (UniqueName: \"kubernetes.io/projected/9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c-kube-api-access-jlksj\") pod \"multus-additional-cni-plugins-lsgx2\" (UID: \"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c\") " pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.809143 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.809118    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v68gz\" (UniqueName: \"kubernetes.io/projected/693c96e8-8277-46a4-8a67-216e9673cdba-kube-api-access-v68gz\") pod \"aws-ebs-csi-driver-node-k4j2p\" (UID: \"693c96e8-8277-46a4-8a67-216e9673cdba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.809421 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.809398    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sck\" (UniqueName: \"kubernetes.io/projected/a3d2cf38-1712-4fcc-b70b-564b399cfa2b-kube-api-access-k6sck\") pod \"node-resolver-mczhb\" (UID: \"a3d2cf38-1712-4fcc-b70b-564b399cfa2b\") " pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.894395 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.894360    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-host\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.894585 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.894412    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-serviceca\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.894585 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.894434    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-host\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.894585 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.894456    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb76t\" (UniqueName: \"kubernetes.io/projected/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-kube-api-access-lb76t\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.895017 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.894968    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-serviceca\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.911079 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.911028    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb76t\" (UniqueName: \"kubernetes.io/projected/9cb20c91-2a09-456c-ae6e-d43b64f3aa93-kube-api-access-lb76t\") pod \"node-ca-k74tn\" (UID: \"9cb20c91-2a09-456c-ae6e-d43b64f3aa93\") " pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:26.954467 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.954414    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal" event={"ID":"8aeda92cfb515398ae0d19a52ccec463","Type":"ContainerStarted","Data":"7320547f3d565f4cd097318f45eb693285d18fa5538c5ab2b0a8820fb1397fb0"}
Jul 08 06:55:26.956236 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.956148    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal" event={"ID":"14c76a2688b1ab5dfa064b62e6726935","Type":"ContainerStarted","Data":"01aac124b13461789e642afac279a3f5cf00217ee437d7206c8f4a1d8ed8d853"}
Jul 08 06:55:26.959778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.959535    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:26.966088 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.965929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mczhb"
Jul 08 06:55:26.975615 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.975596    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p"
Jul 08 06:55:26.983129 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.982735    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lsgx2"
Jul 08 06:55:26.987139 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:26.987113    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d2cf38_1712_4fcc_b70b_564b399cfa2b.slice/crio-10c439ffa91d8868ed8d0d0521cef45c8c7739a68d7ae1cd2bbbf3d59f189a90 WatchSource:0}: Error finding container 10c439ffa91d8868ed8d0d0521cef45c8c7739a68d7ae1cd2bbbf3d59f189a90: Status 404 returned error can't find the container with id 10c439ffa91d8868ed8d0d0521cef45c8c7739a68d7ae1cd2bbbf3d59f189a90
Jul 08 06:55:26.992791 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:26.992444    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tc5xm"
Jul 08 06:55:26.996000 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:26.995970    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693c96e8_8277_46a4_8a67_216e9673cdba.slice/crio-224632ca938d032bb5d40318a20959a0566fc97266671bed78a0ee2ea620745d WatchSource:0}: Error finding container 224632ca938d032bb5d40318a20959a0566fc97266671bed78a0ee2ea620745d: Status 404 returned error can't find the container with id 224632ca938d032bb5d40318a20959a0566fc97266671bed78a0ee2ea620745d
Jul 08 06:55:26.999555 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:26.999528    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e84c25b_8b4c_49b5_87a1_4b1f1f10d42c.slice/crio-87e33f3b713ca92c6a733a5eade87238760ad2eebe172d595259adcbe2819123 WatchSource:0}: Error finding container 87e33f3b713ca92c6a733a5eade87238760ad2eebe172d595259adcbe2819123: Status 404 returned error can't find the container with id 87e33f3b713ca92c6a733a5eade87238760ad2eebe172d595259adcbe2819123
Jul 08 06:55:27.000643 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.000463    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lkzph"
Jul 08 06:55:27.006717 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:27.006687    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79cf45f_e70a_412a_8b70_f1b8066a066d.slice/crio-183d7fb15d8a706a26c408c605b7f1fdcab4c678aa2a53430ff661187ff00bab WatchSource:0}: Error finding container 183d7fb15d8a706a26c408c605b7f1fdcab4c678aa2a53430ff661187ff00bab: Status 404 returned error can't find the container with id 183d7fb15d8a706a26c408c605b7f1fdcab4c678aa2a53430ff661187ff00bab
Jul 08 06:55:27.011559 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.011536    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:27.018805 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.018655    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bb65s"
Jul 08 06:55:27.021740 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:27.021717    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8520e981_4ad3_4258_bd34_b9eb7bdcecd4.slice/crio-a11490e9fd5a3ecde324f632b4c4c53b384b93314d08d62756dca5698888bafb WatchSource:0}: Error finding container a11490e9fd5a3ecde324f632b4c4c53b384b93314d08d62756dca5698888bafb: Status 404 returned error can't find the container with id a11490e9fd5a3ecde324f632b4c4c53b384b93314d08d62756dca5698888bafb
Jul 08 06:55:27.025700 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.025678    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k74tn"
Jul 08 06:55:27.027771 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:27.027744    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e000e0_f8fc_4045_84cb_ae1a8b8cae39.slice/crio-6162193ab1258685ebf364c99f465674b90e20f26fedbc45ffe60674f840d2e4 WatchSource:0}: Error finding container 6162193ab1258685ebf364c99f465674b90e20f26fedbc45ffe60674f840d2e4: Status 404 returned error can't find the container with id 6162193ab1258685ebf364c99f465674b90e20f26fedbc45ffe60674f840d2e4
Jul 08 06:55:27.039212 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:27.039190    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod679705c9_0c44_4084_9bae_366fe27be3e7.slice/crio-c5e8784536b73b24894e64c52314c56ae9f443187c88f2e2aefda5690416294b WatchSource:0}: Error finding container c5e8784536b73b24894e64c52314c56ae9f443187c88f2e2aefda5690416294b: Status 404 returned error can't find the container with id c5e8784536b73b24894e64c52314c56ae9f443187c88f2e2aefda5690416294b
Jul 08 06:55:27.043686 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:55:27.043654    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb20c91_2a09_456c_ae6e_d43b64f3aa93.slice/crio-296c75cfd53914f768ab55f23670189313d90fdda167e6dab96fc39546f6c7a1 WatchSource:0}: Error finding container 296c75cfd53914f768ab55f23670189313d90fdda167e6dab96fc39546f6c7a1: Status 404 returned error can't find the container with id 296c75cfd53914f768ab55f23670189313d90fdda167e6dab96fc39546f6c7a1
Jul 08 06:55:27.298104 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.298074    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:27.298275 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.298114    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:27.298275 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.298237    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:27.298425 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.298239    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:27.298425 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.298297    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:55:28.298271437 +0000 UTC m=+4.085107733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:27.298425 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.298330    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:55:28.298315965 +0000 UTC m=+4.085152261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:27.399517 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.399479    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:27.399728 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.399704    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:27.399810 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.399740    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:27.399810 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.399754    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:27.399897 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:27.399819    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:55:28.399796608 +0000 UTC m=+4.186632928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:27.734768 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.734719    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 06:50:27 +0000 UTC" deadline="2027-12-06 21:38:49.515798148 +0000 UTC"
Jul 08 06:55:27.734768 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.734764    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12398h43m21.781038935s"
Jul 08 06:55:27.963329 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.963184    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qpzwf" event={"ID":"91e000e0-f8fc-4045-84cb-ae1a8b8cae39","Type":"ContainerStarted","Data":"6162193ab1258685ebf364c99f465674b90e20f26fedbc45ffe60674f840d2e4"}
Jul 08 06:55:27.965053 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.964995    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lkzph" event={"ID":"8520e981-4ad3-4258-bd34-b9eb7bdcecd4","Type":"ContainerStarted","Data":"a11490e9fd5a3ecde324f632b4c4c53b384b93314d08d62756dca5698888bafb"}
Jul 08 06:55:27.967508 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.967480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p" event={"ID":"693c96e8-8277-46a4-8a67-216e9673cdba","Type":"ContainerStarted","Data":"224632ca938d032bb5d40318a20959a0566fc97266671bed78a0ee2ea620745d"}
Jul 08 06:55:27.977955 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.977922    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mczhb" event={"ID":"a3d2cf38-1712-4fcc-b70b-564b399cfa2b","Type":"ContainerStarted","Data":"10c439ffa91d8868ed8d0d0521cef45c8c7739a68d7ae1cd2bbbf3d59f189a90"}
Jul 08 06:55:27.980160 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.980129    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"f9acfd947ec3a560630186e4290a30ea17827fcee7404d207ac36232daa06d1c"}
Jul 08 06:55:27.982228 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.982168    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k74tn" event={"ID":"9cb20c91-2a09-456c-ae6e-d43b64f3aa93","Type":"ContainerStarted","Data":"296c75cfd53914f768ab55f23670189313d90fdda167e6dab96fc39546f6c7a1"}
Jul 08 06:55:27.984069 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.984040    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bb65s" event={"ID":"679705c9-0c44-4084-9bae-366fe27be3e7","Type":"ContainerStarted","Data":"c5e8784536b73b24894e64c52314c56ae9f443187c88f2e2aefda5690416294b"}
Jul 08 06:55:27.986348 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.986283    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tc5xm" event={"ID":"b79cf45f-e70a-412a-8b70-f1b8066a066d","Type":"ContainerStarted","Data":"183d7fb15d8a706a26c408c605b7f1fdcab4c678aa2a53430ff661187ff00bab"}
Jul 08 06:55:27.988060 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:27.987995    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerStarted","Data":"87e33f3b713ca92c6a733a5eade87238760ad2eebe172d595259adcbe2819123"}
Jul 08 06:55:28.306261 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.306150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:28.306261 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.306212    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:28.306518 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.306396    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:28.306518 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.306472    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:55:30.306449027 +0000 UTC m=+6.093285351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:28.306870 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.306774    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:28.306870 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.306840    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:55:30.306821027 +0000 UTC m=+6.093657328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:28.407362 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.407296    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:28.407600 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.407549    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:28.407600 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.407574    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:28.407724 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.407612    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:28.407724 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.407677    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:55:30.407656652 +0000 UTC m=+6.194492970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:28.735358 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.735284    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 06:50:27 +0000 UTC" deadline="2028-03-17 14:07:18.567414082 +0000 UTC"
Jul 08 06:55:28.735358 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.735324    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14839h11m49.832094499s"
Jul 08 06:55:28.945244 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.944839    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:28.945244 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.944839    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:28.945244 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:28.944987    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:28.945244 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.944978    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:28.945244 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.945070    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:28.945244 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:28.945146    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:30.327886 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:30.327844    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:30.328423 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:30.327905    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:30.328423 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.328053    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:30.328423 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.328117    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:55:34.328097534 +0000 UTC m=+10.114933837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:30.328655 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.328538    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:30.328655 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.328587    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:55:34.32857199 +0000 UTC m=+10.115408304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:30.428337 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:30.428294    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:30.428544 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.428526    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:30.428617 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.428551    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:30.428617 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.428564    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:30.428763 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.428643    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:55:34.428622554 +0000 UTC m=+10.215458866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:30.945233 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:30.945131    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:30.945233 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:30.945169    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:30.945504 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.945278    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:30.945504 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.945378    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:30.946041 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:30.945720    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:30.946041 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:30.945881    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:32.945704 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:32.944838    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:32.945704 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:32.944993    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:32.945704 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:32.945385    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:32.945704 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:32.945482    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:32.945704 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:32.945566    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:32.945704 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:32.945661    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:34.361453 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:34.361361    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:34.361453 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:34.361415    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:34.361975 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.361577    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:34.361975 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.361652    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:55:42.361632211 +0000 UTC m=+18.148468516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:34.361975 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.361581    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:34.361975 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.361699    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:55:42.361688491 +0000 UTC m=+18.148524785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:34.462088 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:34.462045    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:34.462279 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.462250    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:34.462279 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.462279    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:34.462392 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.462295    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:34.462392 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.462353    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:55:42.462334205 +0000 UTC m=+18.249170501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:34.946838 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:34.946144    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:34.946838 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.946268    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:34.946838 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:34.946702    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:34.946838 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.946796    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:34.947195 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:34.947004    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:34.947195 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:34.947081    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:36.944730 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:36.944696    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:36.945188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:36.944698    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:36.945188 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:36.944834    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:36.945188 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:36.944936    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:36.945188 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:36.944985    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:36.945188 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:36.945048    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:38.945539 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:38.945339    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:38.945963 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:38.945366    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:38.945963 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:38.945666    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:38.945963 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:38.945741    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:38.945963 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:38.945339    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:38.945963 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:38.945838    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:40.944889 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:40.944854    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:40.945336 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:40.944857    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:40.945336 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:40.944989    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:40.945336 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:40.944854    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:40.945336 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:40.945107    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:40.945336 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:40.945161    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:42.424688 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:42.424651    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:42.424688 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:42.424700    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:42.425240 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.424817    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:42.425240 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.424836    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:42.425240 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.424871    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:55:58.424852606 +0000 UTC m=+34.211688903 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:42.425240 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.424912    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:55:58.42489608 +0000 UTC m=+34.211732377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:42.525674 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:42.525637    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:42.525855 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.525777    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:42.525855 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.525790    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:42.525855 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.525799    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:42.525855 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.525844    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:55:58.525831461 +0000 UTC m=+34.312667755 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:42.945167 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:42.945125    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:42.945357 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.945244    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:42.945357 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:42.945127    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:42.945357 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:42.945278    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:42.945357 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.945347    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:42.945556 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:42.945415    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:44.945100 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:44.945062    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:44.945573 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:44.945170    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:44.945573 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:44.945251    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:44.945573 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:44.945352    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:44.945573 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:44.945395    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:44.945573 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:44.945479    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:46.944609 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:46.944582    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:46.944609 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:46.944603    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:46.945027 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:46.944591    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:46.945027 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:46.944713    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:46.945027 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:46.944811    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:46.945027 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:46.944886    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:47.043432 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.043329    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"d9df00da992c91ac5d739bf086053d202235996500dd7b6c8e00f3d818da9539"}
Jul 08 06:55:47.051445 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.051307    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal" event={"ID":"8aeda92cfb515398ae0d19a52ccec463","Type":"ContainerStarted","Data":"c0d459d3d799fe91a136caee2df10941e077994d22a94649bd438482a553c00e"}
Jul 08 06:55:47.056396 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.056229    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bb65s" event={"ID":"679705c9-0c44-4084-9bae-366fe27be3e7","Type":"ContainerStarted","Data":"2e673c80a8172d572f79a942a6d2ddcbfd53603a141bbacb3422e70c8e2d0b6a"}
Jul 08 06:55:47.060398 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.060160    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lkzph" event={"ID":"8520e981-4ad3-4258-bd34-b9eb7bdcecd4","Type":"ContainerStarted","Data":"e8be5f19ebb9622c9cb3f7d05a4889e70e7be18b25ae7216b1a69cf0970fe15d"}
Jul 08 06:55:47.101139 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.101051    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-155.ec2.internal" podStartSLOduration=21.101028244 podStartE2EDuration="21.101028244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:47.082927646 +0000 UTC m=+22.869763966" watchObservedRunningTime="2026-07-08 06:55:47.101028244 +0000 UTC m=+22.887864561"
Jul 08 06:55:47.119537 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.119424    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lkzph" podStartSLOduration=2.492359097 podStartE2EDuration="22.11940457s" totalImagesPullingTime="19.627045473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:47.100686161 +0000 UTC m=+22.887522504" watchObservedRunningTime="2026-07-08 06:55:47.11940457 +0000 UTC m=+22.906240894"
Jul 08 06:55:47.120748 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:47.120690    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bb65s" podStartSLOduration=2.524184446 podStartE2EDuration="22.120673286s" totalImagesPullingTime="19.59648884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:47.120612705 +0000 UTC m=+22.907449009" watchObservedRunningTime="2026-07-08 06:55:47.120673286 +0000 UTC m=+22.907509604"
Jul 08 06:55:48.063209 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.063050    2585 generic.go:356] "Generic (PLEG): container finished" podID="14c76a2688b1ab5dfa064b62e6726935" containerID="38966489f449920ca8cfae198dfd5b488276b8cf9c3db6fc71fd6b693d491d15" exitCode=0
Jul 08 06:55:48.063985 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.063131    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal" event={"ID":"14c76a2688b1ab5dfa064b62e6726935","Type":"ContainerDied","Data":"38966489f449920ca8cfae198dfd5b488276b8cf9c3db6fc71fd6b693d491d15"}
Jul 08 06:55:48.064780 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.064755    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qpzwf" event={"ID":"91e000e0-f8fc-4045-84cb-ae1a8b8cae39","Type":"ContainerStarted","Data":"9c664256cdde801652c6a5a441a3d91486d89eb8bed05cf32e84e1230a16164a"}
Jul 08 06:55:48.066238 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.066217    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p" event={"ID":"693c96e8-8277-46a4-8a67-216e9673cdba","Type":"ContainerStarted","Data":"cc9ca885c7c3af337761a9d613278650ba2ac059f77bcc8b2e522dbff67ae43e"}
Jul 08 06:55:48.067351 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.067333    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mczhb" event={"ID":"a3d2cf38-1712-4fcc-b70b-564b399cfa2b","Type":"ContainerStarted","Data":"a405f6ca40f9b6b9fb66c8078c2d7464b0336d7b1ac492ed69d38fec33231b24"}
Jul 08 06:55:48.070183 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.070162    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"dd9c3d44d699b32bcab10bacd1affda4f3f4db596522cc17399d7309e25859be"}
Jul 08 06:55:48.070260 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.070237    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"d4d44d16e553a369363c74fbcc8d690334d26d6ae2775d866a449e945c4e4f9c"}
Jul 08 06:55:48.070301 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.070289    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"016f265c9355eeda78f4e5b0b5281933f69e6ccbf841aebcefab3d7e54cfb85d"}
Jul 08 06:55:48.070366 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.070353    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"5c9e4c4707ace66b593b2c12981783fa543ac97a307ca6fd523e867c018a5444"}
Jul 08 06:55:48.070414 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.070389    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"3111e490426b0d9f33887a174fb2cc03c1fc24dfd56fd2facb1943f2edf1c4a0"}
Jul 08 06:55:48.071843 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.071818    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k74tn" event={"ID":"9cb20c91-2a09-456c-ae6e-d43b64f3aa93","Type":"ContainerStarted","Data":"a802450839776fcd7991fb71551ecaab2597719d2fb49f42df98c8aad34cfb35"}
Jul 08 06:55:48.074134 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.074112    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tc5xm" event={"ID":"b79cf45f-e70a-412a-8b70-f1b8066a066d","Type":"ContainerStarted","Data":"af7123b93025a61fd9e194f9f618a20b227ca036bfa0b223feb14fc90e92ff88"}
Jul 08 06:55:48.075341 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.075323    2585 generic.go:356] "Generic (PLEG): container finished" podID="9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c" containerID="922b95a40248056afb3e0f82a7569bb35e2706bf24cc49605308509f3332c0d9" exitCode=0
Jul 08 06:55:48.075413 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.075394    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerDied","Data":"922b95a40248056afb3e0f82a7569bb35e2706bf24cc49605308509f3332c0d9"}
Jul 08 06:55:48.094991 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.094945    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mczhb" podStartSLOduration=3.511957562 podStartE2EDuration="23.094914873s" totalImagesPullingTime="19.582957311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:48.094870227 +0000 UTC m=+23.881706557" watchObservedRunningTime="2026-07-08 06:55:48.094914873 +0000 UTC m=+23.881751191"
Jul 08 06:55:48.148435 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.148377    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qpzwf" podStartSLOduration=3.543157844 podStartE2EDuration="23.148361474s" totalImagesPullingTime="19.60520363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:48.148026811 +0000 UTC m=+23.934863129" watchObservedRunningTime="2026-07-08 06:55:48.148361474 +0000 UTC m=+23.935197790"
Jul 08 06:55:48.148727 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.148696    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tc5xm" podStartSLOduration=3.584018139 podStartE2EDuration="23.148688134s" totalImagesPullingTime="19.564669995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:48.133428639 +0000 UTC m=+23.920264955" watchObservedRunningTime="2026-07-08 06:55:48.148688134 +0000 UTC m=+23.935524461"
Jul 08 06:55:48.162448 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.162411    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k74tn" podStartSLOduration=3.57281739 podStartE2EDuration="23.162393106s" totalImagesPullingTime="19.589575716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:48.162086669 +0000 UTC m=+23.948922986" watchObservedRunningTime="2026-07-08 06:55:48.162393106 +0000 UTC m=+23.949229448"
Jul 08 06:55:48.283822 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.283791    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:48.352610 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.352454    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:48.945144 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.945117    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:48.945298 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.945156    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:48.945298 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:48.945250    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:48.945417 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:48.945319    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:48.945417 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.945392    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:48.945522 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:48.945483    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:48.966214 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:48.966187    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 06:55:49.080298 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:49.080263    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal" event={"ID":"14c76a2688b1ab5dfa064b62e6726935","Type":"ContainerStarted","Data":"ac4a8641ce2769f1c9f2775ec6526fbb2ab51057107eb0e7e8de1aa6c7208033"}
Jul 08 06:55:49.082289 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:49.082260    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p" event={"ID":"693c96e8-8277-46a4-8a67-216e9673cdba","Type":"ContainerStarted","Data":"19e3808a75293b927cf0d9192668ea222f34de087c3ecddfa7c03505385b69e1"}
Jul 08 06:55:49.096509 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:49.096412    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-155.ec2.internal" podStartSLOduration=23.096393532 podStartE2EDuration="23.096393532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:49.095871352 +0000 UTC m=+24.882707669" watchObservedRunningTime="2026-07-08 06:55:49.096393532 +0000 UTC m=+24.883229850"
Jul 08 06:55:49.812437 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:49.812337    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T06:55:48.966210428Z","UUID":"b1a5a7a5-7321-4949-8c81-1762369e0c02","Handler":null,"Name":"","Endpoint":""}
Jul 08 06:55:49.814023 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:49.814005    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 06:55:49.814139 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:49.814046    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 06:55:50.088628 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:50.088587    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"d77c61e2c4b5e83e9c869783db2f64b6a3c776898850cc2beca24161ef591ed9"}
Jul 08 06:55:50.604387 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:50.604347    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:50.605053 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:50.605033    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:50.944959 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:50.944925    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:50.945140 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:50.944925    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:50.945140 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:50.945043    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:50.945140 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:50.945109    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:50.945140 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:50.944924    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:50.945320 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:50.945188    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:51.050580 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.050534    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:51.092637 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.092591    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p" event={"ID":"693c96e8-8277-46a4-8a67-216e9673cdba","Type":"ContainerStarted","Data":"9fa8fab953c5156b57e13841afa47c33836f2d11abbeaae125dd2a0c48c4b59d"}
Jul 08 06:55:51.093094 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.092739    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:51.093208 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.093193    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qpzwf"
Jul 08 06:55:51.112266 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.112202    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4j2p" podStartSLOduration=4.02691105 podStartE2EDuration="26.112185948s" totalImagesPullingTime="22.085274898s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:51.110469415 +0000 UTC m=+26.897305733" watchObservedRunningTime="2026-07-08 06:55:51.112185948 +0000 UTC m=+26.899022264"
Jul 08 06:55:51.122536 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.122494    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:51.236219 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:51.236113    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:52.883572 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:52.883358    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:52.944713 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:52.944689    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:52.944713 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:52.944702    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:52.944832 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:52.944698    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:52.944863 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:52.944821    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:52.944863 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:52.944853    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:52.944989 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:52.944963    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:53.100356 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.100321    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" event={"ID":"eb5ab310-b866-4cda-86fc-689295b20301","Type":"ContainerStarted","Data":"28f1af7b5e4632292a406828db015c3523e915a24dd20d6622a23b49a81f5460"}
Jul 08 06:55:53.100760 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.100669    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:53.102157 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.102130    2585 generic.go:356] "Generic (PLEG): container finished" podID="9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c" containerID="ef068789bf15671244323cb8e15652406cac358dd793a6ecd99c46bbe4abdf3c" exitCode=0
Jul 08 06:55:53.102258 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.102196    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerDied","Data":"ef068789bf15671244323cb8e15652406cac358dd793a6ecd99c46bbe4abdf3c"}
Jul 08 06:55:53.117576 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.117536    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:53.131834 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.131790    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd" podStartSLOduration=8.176031436 podStartE2EDuration="28.131778047s" totalImagesPullingTime="19.955746611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:53.131301443 +0000 UTC m=+28.918137760" watchObservedRunningTime="2026-07-08 06:55:53.131778047 +0000 UTC m=+28.918614363"
Jul 08 06:55:53.430227 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.430186    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:53.532349 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:53.532309    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:54.105888 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.105857    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:54.106357 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.105946    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:54.121897 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.121865    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:55:54.697810 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.697491    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xpq58"]
Jul 08 06:55:54.698007 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.697854    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:54.698007 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:54.697967    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:54.700096 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.700069    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j48lr"]
Jul 08 06:55:54.700292 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.700278    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlrcx"]
Jul 08 06:55:54.700662 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.700631    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:54.700820 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:54.700786    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:54.700879 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:54.700869    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:54.700989 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:54.700967    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:55.109915 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:55.109833    2585 generic.go:356] "Generic (PLEG): container finished" podID="9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c" containerID="4f084d01080b0eb2a0448be80abdab007c76af120acb381f1700d16d36960c58" exitCode=0
Jul 08 06:55:55.110304 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:55.109925    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerDied","Data":"4f084d01080b0eb2a0448be80abdab007c76af120acb381f1700d16d36960c58"}
Jul 08 06:55:55.945002 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:55.944971    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:55.945002 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:55.944996    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:55.945186 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:55.945073    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:55.945226 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:55.945181    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:56.944977 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:56.944941    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:56.945313 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:56.945045    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:57.119324 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:57.119226    2585 generic.go:356] "Generic (PLEG): container finished" podID="9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c" containerID="bd6e4b4724970339e7d842f2e64f6e656e9341aec539c62b979b9eb11adc38f9" exitCode=0
Jul 08 06:55:57.119324 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:57.119278    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerDied","Data":"bd6e4b4724970339e7d842f2e64f6e656e9341aec539c62b979b9eb11adc38f9"}
Jul 08 06:55:57.945363 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:57.945143    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:57.945817 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:57.945155    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:57.945817 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:57.945494    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlrcx" podUID="6de506fd-0787-407c-97e3-349973298b4f"
Jul 08 06:55:57.945817 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:57.945595    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpq58" podUID="fcd81370-ee72-4e93-9089-44790495c67b"
Jul 08 06:55:58.438852 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:58.438811    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:58.439011 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:58.438859    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:58.439011 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.438976    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:58.439115 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.439042    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs podName:6de506fd-0787-407c-97e3-349973298b4f nodeName:}" failed. No retries permitted until 2026-07-08 06:56:30.439023013 +0000 UTC m=+66.225859313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs") pod "network-metrics-daemon-mlrcx" (UID: "6de506fd-0787-407c-97e3-349973298b4f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:58.439115 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.438976    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:58.439195 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.439126    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret podName:fcd81370-ee72-4e93-9089-44790495c67b nodeName:}" failed. No retries permitted until 2026-07-08 06:56:30.439107016 +0000 UTC m=+66.225943331 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret") pod "global-pull-secret-syncer-xpq58" (UID: "fcd81370-ee72-4e93-9089-44790495c67b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:58.540180 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:58.540144    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:58.540363 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.540302    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:58.540363 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.540322    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:58.540363 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.540333    2585 projected.go:196] Error preparing data for projected volume kube-api-access-699rd for pod openshift-network-diagnostics/network-check-target-j48lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:58.540527 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.540388    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd podName:484faa4e-d43a-4e7a-9f26-8373ae6cafde nodeName:}" failed. No retries permitted until 2026-07-08 06:56:30.540368912 +0000 UTC m=+66.327205227 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-699rd" (UniqueName: "kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd") pod "network-check-target-j48lr" (UID: "484faa4e-d43a-4e7a-9f26-8373ae6cafde") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:58.944577 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:58.944529    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:55:58.944754 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:58.944685    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j48lr" podUID="484faa4e-d43a-4e7a-9f26-8373ae6cafde"
Jul 08 06:55:59.487618 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.487589    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-155.ec2.internal" event="NodeReady"
Jul 08 06:55:59.488028 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.487737    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 06:55:59.529197 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.529159    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5ff5888c74-8g62g"]
Jul 08 06:55:59.533391 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.533362    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.538039 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.538017    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 06:55:59.538165 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.538090    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4pqkf\""
Jul 08 06:55:59.538625 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.538607    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 06:55:59.538719 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.538649    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 06:55:59.542634 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.542546    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7lqgk"]
Jul 08 06:55:59.548376 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.548084    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ll8tp"]
Jul 08 06:55:59.548748 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.548722    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.549160 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.549133    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 06:55:59.551611 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.551587    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 06:55:59.551703 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.551617    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 06:55:59.551768 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.551748    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pz4bn\""
Jul 08 06:55:59.553425 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.553404    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ff5888c74-8g62g"]
Jul 08 06:55:59.553592 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.553503    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.556739 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.556716    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 06:55:59.556826 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.556760    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 06:55:59.556878 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.556825    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qfqgp\""
Jul 08 06:55:59.557360 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.557343    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 06:55:59.560711 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.560691    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lqgk"]
Jul 08 06:55:59.563905 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.563882    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ll8tp"]
Jul 08 06:55:59.575214 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.575190    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fcfnd"]
Jul 08 06:55:59.579147 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.579125    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.582399 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.582376    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 06:55:59.582399 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.582389    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 06:55:59.582593 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.582378    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 06:55:59.582646 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.582622    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f5bjk\""
Jul 08 06:55:59.583236 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.583219    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 06:55:59.593612 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.593590    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fcfnd"]
Jul 08 06:55:59.648451 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648419    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-image-registry-private-configuration\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648451 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648459    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648496    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-ca-trust-extracted\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648527    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-certificates\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648550    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-installation-pull-secrets\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-bound-sa-token\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648615    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-trusted-ca\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648679    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn46c\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-kube-api-access-mn46c\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648718    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828fffaa-c27d-4036-bb01-bd2b404fd358-tmp-dir\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648754    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19ab5bed-e37e-413e-a574-003de2a693b9-cert\") pod \"ingress-canary-ll8tp\" (UID: \"19ab5bed-e37e-413e-a574-003de2a693b9\") " pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.648779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648774    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.649085 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648806    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-999dq\" (UniqueName: \"kubernetes.io/projected/19ab5bed-e37e-413e-a574-003de2a693b9-kube-api-access-999dq\") pod \"ingress-canary-ll8tp\" (UID: \"19ab5bed-e37e-413e-a574-003de2a693b9\") " pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.649085 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648825    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/828fffaa-c27d-4036-bb01-bd2b404fd358-config-volume\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.649085 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.648845    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8sc8\" (UniqueName: \"kubernetes.io/projected/828fffaa-c27d-4036-bb01-bd2b404fd358-kube-api-access-h8sc8\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.749765 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749731    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/737866f6-e412-4126-916b-ff22685fc6b1-crio-socket\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749770    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/737866f6-e412-4126-916b-ff22685fc6b1-data-volume\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749833    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-999dq\" (UniqueName: \"kubernetes.io/projected/19ab5bed-e37e-413e-a574-003de2a693b9-kube-api-access-999dq\") pod \"ingress-canary-ll8tp\" (UID: \"19ab5bed-e37e-413e-a574-003de2a693b9\") " pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749863    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/828fffaa-c27d-4036-bb01-bd2b404fd358-config-volume\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749881    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8sc8\" (UniqueName: \"kubernetes.io/projected/828fffaa-c27d-4036-bb01-bd2b404fd358-kube-api-access-h8sc8\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749900    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749917    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.749998    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-image-registry-private-configuration\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750010 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750015    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750033    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-ca-trust-extracted\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750062    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-certificates\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750087    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-installation-pull-secrets\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750110    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-bound-sa-token\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.750117    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.750137    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ff5888c74-8g62g: secret "image-registry-tls" not found
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-trusted-ca\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750159    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn46c\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-kube-api-access-mn46c\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750181    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828fffaa-c27d-4036-bb01-bd2b404fd358-tmp-dir\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.750218    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls podName:1d3fd240-61f7-4cb8-9fea-407f382f6a1b nodeName:}" failed. No retries permitted until 2026-07-08 06:56:00.250195277 +0000 UTC m=+36.037031575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls") pod "image-registry-5ff5888c74-8g62g" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b") : secret "image-registry-tls" not found
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750241    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnhgm\" (UniqueName: \"kubernetes.io/projected/737866f6-e412-4126-916b-ff22685fc6b1-kube-api-access-hnhgm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750281    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19ab5bed-e37e-413e-a574-003de2a693b9-cert\") pod \"ingress-canary-ll8tp\" (UID: \"19ab5bed-e37e-413e-a574-003de2a693b9\") " pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750311    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.750420 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750422    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828fffaa-c27d-4036-bb01-bd2b404fd358-tmp-dir\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.751004 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.750424    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:59.751004 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750450    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-ca-trust-extracted\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.751004 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750474    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/828fffaa-c27d-4036-bb01-bd2b404fd358-config-volume\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.751004 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.750499    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls podName:828fffaa-c27d-4036-bb01-bd2b404fd358 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:00.250483976 +0000 UTC m=+36.037320278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls") pod "dns-default-7lqgk" (UID: "828fffaa-c27d-4036-bb01-bd2b404fd358") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:59.751004 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.750729    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-certificates\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.751337 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.751312    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-trusted-ca\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.754951 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.754928    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-installation-pull-secrets\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.755069 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.754940    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19ab5bed-e37e-413e-a574-003de2a693b9-cert\") pod \"ingress-canary-ll8tp\" (UID: \"19ab5bed-e37e-413e-a574-003de2a693b9\") " pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.755069 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.754934    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-image-registry-private-configuration\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.762504 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.762479    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-999dq\" (UniqueName: \"kubernetes.io/projected/19ab5bed-e37e-413e-a574-003de2a693b9-kube-api-access-999dq\") pod \"ingress-canary-ll8tp\" (UID: \"19ab5bed-e37e-413e-a574-003de2a693b9\") " pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.763700 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.763680    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-bound-sa-token\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.765823 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.765803    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8sc8\" (UniqueName: \"kubernetes.io/projected/828fffaa-c27d-4036-bb01-bd2b404fd358-kube-api-access-h8sc8\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:55:59.766118 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.766098    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn46c\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-kube-api-access-mn46c\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:55:59.851232 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnhgm\" (UniqueName: \"kubernetes.io/projected/737866f6-e412-4126-916b-ff22685fc6b1-kube-api-access-hnhgm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.851232 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851227    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/737866f6-e412-4126-916b-ff22685fc6b1-crio-socket\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.851440 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851253    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/737866f6-e412-4126-916b-ff22685fc6b1-data-volume\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.851440 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851283    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.851440 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851308    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.851440 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.851405    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:55:59.851440 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.851424    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:59.851688 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.851496    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:00.351472927 +0000 UTC m=+36.138309243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : configmap "kube-rbac-proxy" not found
Jul 08 06:55:59.851688 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:55:59.851517    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:00.351507016 +0000 UTC m=+36.138343327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:59.851688 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851517    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/737866f6-e412-4126-916b-ff22685fc6b1-crio-socket\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.851688 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.851675    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/737866f6-e412-4126-916b-ff22685fc6b1-data-volume\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.864572 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.864536    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnhgm\" (UniqueName: \"kubernetes.io/projected/737866f6-e412-4126-916b-ff22685fc6b1-kube-api-access-hnhgm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:55:59.868511 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.868479    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ll8tp"
Jul 08 06:55:59.944524 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.944480    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:55:59.944723 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.944540    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:55:59.948002 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.947978    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 06:55:59.948132 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.948049    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 06:55:59.948132 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:55:59.948082    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jfr79\""
Jul 08 06:56:00.124687 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.124605    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mczhb_a3d2cf38-1712-4fcc-b70b-564b399cfa2b/dns-node-resolver/0.log"
Jul 08 06:56:00.254389 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.254344    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:00.254593 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.254498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:00.254593 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.254511    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:56:00.254718 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.254606    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls podName:828fffaa-c27d-4036-bb01-bd2b404fd358 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:01.254584574 +0000 UTC m=+37.041420884 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls") pod "dns-default-7lqgk" (UID: "828fffaa-c27d-4036-bb01-bd2b404fd358") : secret "dns-default-metrics-tls" not found
Jul 08 06:56:00.254718 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.254629    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:56:00.254718 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.254643    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ff5888c74-8g62g: secret "image-registry-tls" not found
Jul 08 06:56:00.254718 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.254696    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls podName:1d3fd240-61f7-4cb8-9fea-407f382f6a1b nodeName:}" failed. No retries permitted until 2026-07-08 06:56:01.254679044 +0000 UTC m=+37.041515346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls") pod "image-registry-5ff5888c74-8g62g" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b") : secret "image-registry-tls" not found
Jul 08 06:56:00.354896 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.354857    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:00.355080 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.354909    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:00.355080 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.354951    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:56:00.355080 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.355041    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:01.35500914 +0000 UTC m=+37.141845439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : configmap "kube-rbac-proxy" not found
Jul 08 06:56:00.355080 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.355041    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:00.355080 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:00.355082    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:01.355072543 +0000 UTC m=+37.141908837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:00.945228 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.945195    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:56:00.948767 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.948744    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 06:56:00.949138 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.949119    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-97jj5\""
Jul 08 06:56:00.950252 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.950236    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 06:56:00.988844 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.988807    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-m8r24"]
Jul 08 06:56:00.991959 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.991942    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:00.995152 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.995132    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 08 06:56:00.995289 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.995186    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-q7knp\""
Jul 08 06:56:00.995573 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.995538    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 08 06:56:00.995647 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.995541    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 08 06:56:00.995647 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:00.995607    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 08 06:56:01.004128 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.004108    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-m8r24"]
Jul 08 06:56:01.124723 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.124688    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k74tn_9cb20c91-2a09-456c-ae6e-d43b64f3aa93/node-ca/0.log"
Jul 08 06:56:01.162812 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.162783    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-signing-key\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.162812 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.162818    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwts2\" (UniqueName: \"kubernetes.io/projected/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-kube-api-access-cwts2\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.163027 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.162848    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-signing-cabundle\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.263730 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.263690    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:01.263871 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.263740    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:01.263871 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.263758    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-signing-key\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.263871 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.263777    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwts2\" (UniqueName: \"kubernetes.io/projected/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-kube-api-access-cwts2\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.263871 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.263804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-signing-cabundle\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.264034 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.263867    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:56:01.264034 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.263887    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ff5888c74-8g62g: secret "image-registry-tls" not found
Jul 08 06:56:01.264034 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.263946    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls podName:1d3fd240-61f7-4cb8-9fea-407f382f6a1b nodeName:}" failed. No retries permitted until 2026-07-08 06:56:03.263929693 +0000 UTC m=+39.050765987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls") pod "image-registry-5ff5888c74-8g62g" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b") : secret "image-registry-tls" not found
Jul 08 06:56:01.264034 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.263877    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:56:01.264034 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.263978    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls podName:828fffaa-c27d-4036-bb01-bd2b404fd358 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:03.263972406 +0000 UTC m=+39.050808700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls") pod "dns-default-7lqgk" (UID: "828fffaa-c27d-4036-bb01-bd2b404fd358") : secret "dns-default-metrics-tls" not found
Jul 08 06:56:01.264643 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.264617    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-signing-cabundle\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.266572 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.266545    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-signing-key\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.277497 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.277468    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwts2\" (UniqueName: \"kubernetes.io/projected/aa20681e-7cb3-404e-95e1-0b7f0e64df8c-kube-api-access-cwts2\") pod \"service-ca-758f599c5-m8r24\" (UID: \"aa20681e-7cb3-404e-95e1-0b7f0e64df8c\") " pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.301523 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.301497    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-m8r24"
Jul 08 06:56:01.360544 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.360297    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ll8tp"]
Jul 08 06:56:01.363448 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:01.363413    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ab5bed_e37e_413e_a574_003de2a693b9.slice/crio-76b288d2c8ccff5ba69c4828711ca1dcb8028ea160e1e184ed17eb6b673b5e0c WatchSource:0}: Error finding container 76b288d2c8ccff5ba69c4828711ca1dcb8028ea160e1e184ed17eb6b673b5e0c: Status 404 returned error can't find the container with id 76b288d2c8ccff5ba69c4828711ca1dcb8028ea160e1e184ed17eb6b673b5e0c
Jul 08 06:56:01.365961 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.365149    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:01.365961 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.365182    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:01.365961 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.365291    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:56:01.365961 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.365326    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:01.365961 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.365374    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:03.365351852 +0000 UTC m=+39.152188164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : configmap "kube-rbac-proxy" not found
Jul 08 06:56:01.365961 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:01.365392    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:03.365383431 +0000 UTC m=+39.152219728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:01.452914 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:01.452892    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-m8r24"]
Jul 08 06:56:01.456086 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:01.456058    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa20681e_7cb3_404e_95e1_0b7f0e64df8c.slice/crio-801d95f31a286afd34498f2e5c90f390335a5082297347b3f80b3c64f0bf4873 WatchSource:0}: Error finding container 801d95f31a286afd34498f2e5c90f390335a5082297347b3f80b3c64f0bf4873: Status 404 returned error can't find the container with id 801d95f31a286afd34498f2e5c90f390335a5082297347b3f80b3c64f0bf4873
Jul 08 06:56:02.135786 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:02.135699    2585 generic.go:356] "Generic (PLEG): container finished" podID="9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c" containerID="d2561a1dfde0450be4a9b9269b28929f46e045e7e30d0e96f27ae49ce990cd5d" exitCode=0
Jul 08 06:56:02.136382 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:02.136361    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerDied","Data":"d2561a1dfde0450be4a9b9269b28929f46e045e7e30d0e96f27ae49ce990cd5d"}
Jul 08 06:56:02.137380 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:02.137349    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-m8r24" event={"ID":"aa20681e-7cb3-404e-95e1-0b7f0e64df8c","Type":"ContainerStarted","Data":"801d95f31a286afd34498f2e5c90f390335a5082297347b3f80b3c64f0bf4873"}
Jul 08 06:56:02.138822 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:02.138775    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ll8tp" event={"ID":"19ab5bed-e37e-413e-a574-003de2a693b9","Type":"ContainerStarted","Data":"76b288d2c8ccff5ba69c4828711ca1dcb8028ea160e1e184ed17eb6b673b5e0c"}
Jul 08 06:56:03.147391 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:03.147343    2585 generic.go:356] "Generic (PLEG): container finished" podID="9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c" containerID="c1acbf9a182df826b061c00285a205e320d041f667491040dca94c1c68cab008" exitCode=0
Jul 08 06:56:03.147865 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:03.147403    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerDied","Data":"c1acbf9a182df826b061c00285a205e320d041f667491040dca94c1c68cab008"}
Jul 08 06:56:03.282117 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:03.282070    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:03.282298 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:03.282142    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:03.282298 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.282260    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:56:03.282298 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.282283    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ff5888c74-8g62g: secret "image-registry-tls" not found
Jul 08 06:56:03.282460 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.282299    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:56:03.282460 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.282343    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls podName:1d3fd240-61f7-4cb8-9fea-407f382f6a1b nodeName:}" failed. No retries permitted until 2026-07-08 06:56:07.282323069 +0000 UTC m=+43.069159377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls") pod "image-registry-5ff5888c74-8g62g" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b") : secret "image-registry-tls" not found
Jul 08 06:56:03.282460 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.282361    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls podName:828fffaa-c27d-4036-bb01-bd2b404fd358 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:07.282352655 +0000 UTC m=+43.069188956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls") pod "dns-default-7lqgk" (UID: "828fffaa-c27d-4036-bb01-bd2b404fd358") : secret "dns-default-metrics-tls" not found
Jul 08 06:56:03.383509 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:03.383478    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:03.383714 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:03.383517    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:03.383714 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.383640    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:56:03.383714 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.383654    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:03.383714 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.383707    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:07.383691523 +0000 UTC m=+43.170527836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:03.383868 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:03.383727    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:07.38371679 +0000 UTC m=+43.170553084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : configmap "kube-rbac-proxy" not found
Jul 08 06:56:05.154886 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:05.154667    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" event={"ID":"9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c","Type":"ContainerStarted","Data":"be5c154905f68b6540499db6245bd02da836a8ad70c7b555e86650656e74b04f"}
Jul 08 06:56:05.156117 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:05.156092    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-m8r24" event={"ID":"aa20681e-7cb3-404e-95e1-0b7f0e64df8c","Type":"ContainerStarted","Data":"2880adbe0e6ea2796de1419c67cb2f8522779563d89fedff746920052e073b78"}
Jul 08 06:56:05.157461 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:05.157434    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ll8tp" event={"ID":"19ab5bed-e37e-413e-a574-003de2a693b9","Type":"ContainerStarted","Data":"6c997d24c6e1c15bbff4dca00f6e65645fceffc5d379c76c3823941a590f7077"}
Jul 08 06:56:05.180879 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:05.180836    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lsgx2" podStartSLOduration=9.424043407 podStartE2EDuration="40.18082254s" totalImagesPullingTime="30.756779133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:05.180363986 +0000 UTC m=+40.967200327" watchObservedRunningTime="2026-07-08 06:56:05.18082254 +0000 UTC m=+40.967658856"
Jul 08 06:56:05.198298 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:05.198247    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-m8r24" podStartSLOduration=2.149062081 podStartE2EDuration="5.198228163s" totalImagesPullingTime="3.049166082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:05.197449831 +0000 UTC m=+40.984286147" watchObservedRunningTime="2026-07-08 06:56:05.198228163 +0000 UTC m=+40.985064460"
Jul 08 06:56:07.317923 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:07.317891    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:07.318506 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:07.317989    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:07.318506 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.318040    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:56:07.318506 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.318073    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:56:07.318506 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.318083    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ff5888c74-8g62g: secret "image-registry-tls" not found
Jul 08 06:56:07.318506 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.318111    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls podName:828fffaa-c27d-4036-bb01-bd2b404fd358 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:15.318091109 +0000 UTC m=+51.104927403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls") pod "dns-default-7lqgk" (UID: "828fffaa-c27d-4036-bb01-bd2b404fd358") : secret "dns-default-metrics-tls" not found
Jul 08 06:56:07.318506 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.318130    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls podName:1d3fd240-61f7-4cb8-9fea-407f382f6a1b nodeName:}" failed. No retries permitted until 2026-07-08 06:56:15.318121193 +0000 UTC m=+51.104957489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls") pod "image-registry-5ff5888c74-8g62g" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b") : secret "image-registry-tls" not found
Jul 08 06:56:07.418331 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:07.418293    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:07.418331 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:07.418333    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:07.418538 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.418422    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:56:07.418538 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.418501    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:15.418479325 +0000 UTC m=+51.205315622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : configmap "kube-rbac-proxy" not found
Jul 08 06:56:07.418538 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.418424    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:07.418671 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:07.418553    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:15.418541119 +0000 UTC m=+51.205377412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:15.377006 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.376959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:15.377401 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.377027    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:15.380963 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.380923    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/828fffaa-c27d-4036-bb01-bd2b404fd358-metrics-tls\") pod \"dns-default-7lqgk\" (UID: \"828fffaa-c27d-4036-bb01-bd2b404fd358\") " pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:15.380963 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.380961    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"image-registry-5ff5888c74-8g62g\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") " pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:15.444799 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.444767    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:15.461947 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.461926    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:15.478147 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.478127    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:15.478260 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.478155    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:15.478333 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:15.478318    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:15.478386 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:15.478378    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls podName:737866f6-e412-4126-916b-ff22685fc6b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:31.478360976 +0000 UTC m=+67.265197284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcfnd" (UID: "737866f6-e412-4126-916b-ff22685fc6b1") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:56:15.478942 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.478920    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/737866f6-e412-4126-916b-ff22685fc6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:15.587272 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.587225    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ll8tp" podStartSLOduration=13.452543577 podStartE2EDuration="16.587208392s" totalImagesPullingTime="3.134664815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:05.216729613 +0000 UTC m=+41.003565930" watchObservedRunningTime="2026-07-08 06:56:15.587208392 +0000 UTC m=+51.374044708"
Jul 08 06:56:15.587979 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.587959    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ff5888c74-8g62g"]
Jul 08 06:56:15.591830 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:15.591804    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3fd240_61f7_4cb8_9fea_407f382f6a1b.slice/crio-3f1001c7f2c08419d35ca99ae6e39693f4a9498018bec125d84b36ae720bb4e5 WatchSource:0}: Error finding container 3f1001c7f2c08419d35ca99ae6e39693f4a9498018bec125d84b36ae720bb4e5: Status 404 returned error can't find the container with id 3f1001c7f2c08419d35ca99ae6e39693f4a9498018bec125d84b36ae720bb4e5
Jul 08 06:56:15.600802 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:15.600780    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lqgk"]
Jul 08 06:56:15.604164 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:15.604147    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828fffaa_c27d_4036_bb01_bd2b404fd358.slice/crio-e48a33c91585862bb97800df42c6e4bc35595ed5035ea59c4358c415bdc7d24e WatchSource:0}: Error finding container e48a33c91585862bb97800df42c6e4bc35595ed5035ea59c4358c415bdc7d24e: Status 404 returned error can't find the container with id e48a33c91585862bb97800df42c6e4bc35595ed5035ea59c4358c415bdc7d24e
Jul 08 06:56:16.185226 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:16.185189    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lqgk" event={"ID":"828fffaa-c27d-4036-bb01-bd2b404fd358","Type":"ContainerStarted","Data":"e48a33c91585862bb97800df42c6e4bc35595ed5035ea59c4358c415bdc7d24e"}
Jul 08 06:56:16.186852 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:16.186790    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" event={"ID":"1d3fd240-61f7-4cb8-9fea-407f382f6a1b","Type":"ContainerStarted","Data":"35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d"}
Jul 08 06:56:16.186852 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:16.186839    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" event={"ID":"1d3fd240-61f7-4cb8-9fea-407f382f6a1b","Type":"ContainerStarted","Data":"3f1001c7f2c08419d35ca99ae6e39693f4a9498018bec125d84b36ae720bb4e5"}
Jul 08 06:56:16.187052 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:16.186935    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:16.207795 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:16.207731    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" podStartSLOduration=26.20771199 podStartE2EDuration="26.20771199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:16.206927162 +0000 UTC m=+51.993763480" watchObservedRunningTime="2026-07-08 06:56:16.20771199 +0000 UTC m=+51.994548307"
Jul 08 06:56:18.194290 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:18.194251    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lqgk" event={"ID":"828fffaa-c27d-4036-bb01-bd2b404fd358","Type":"ContainerStarted","Data":"cef02a70e6bab5097dada7ca8e20d93026a18bf9033e294785a09afc3c4916ea"}
Jul 08 06:56:18.194762 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:18.194302    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lqgk" event={"ID":"828fffaa-c27d-4036-bb01-bd2b404fd358","Type":"ContainerStarted","Data":"8835f2ac8e496ee8cbd5f28f6e6bb3f1231801ac649fa0b8709a5e9cf08d6655"}
Jul 08 06:56:18.194762 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:18.194380    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:18.211909 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:18.211839    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7lqgk" podStartSLOduration=17.036551053 podStartE2EDuration="19.211819949s" totalImagesPullingTime="2.175268896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:18.211188845 +0000 UTC m=+53.998025162" watchObservedRunningTime="2026-07-08 06:56:18.211819949 +0000 UTC m=+53.998656263"
Jul 08 06:56:20.993738 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.993708    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"]
Jul 08 06:56:20.995856 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.995839    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"]
Jul 08 06:56:20.995988 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.995972    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:20.997673 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.997653    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"]
Jul 08 06:56:20.997878 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.997859    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:20.999696 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.999673    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:20.999906 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:20.999875    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 08 06:56:21.000171 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.000154    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hq7xz\""
Jul 08 06:56:21.000270 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.000255    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 08 06:56:21.005310 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005284    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 08 06:56:21.005500 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005442    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 08 06:56:21.005640 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005495    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 08 06:56:21.005747 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005728    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 08 06:56:21.005805 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005750    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 08 06:56:21.005805 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005776    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 08 06:56:21.005994 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.005968    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 08 06:56:21.010969 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.010946    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"]
Jul 08 06:56:21.011863 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.011846    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"]
Jul 08 06:56:21.012779 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.012759    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"]
Jul 08 06:56:21.021389 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rc2n\" (UniqueName: \"kubernetes.io/projected/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-kube-api-access-2rc2n\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.021496 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021402    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.021496 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021425    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99fsj\" (UniqueName: \"kubernetes.io/projected/77dc5200-5f81-41ed-a241-a463bda8be58-kube-api-access-99fsj\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.021496 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021457    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-tmp\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.021496 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021491    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6967dbd884-9vwkl\" (UID: \"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.021649 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021510    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-hub\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.021649 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021535    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.021649 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021578    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m24vn\" (UniqueName: \"kubernetes.io/projected/0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec-kube-api-access-m24vn\") pod \"managed-serviceaccount-addon-agent-6967dbd884-9vwkl\" (UID: \"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.021649 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021622    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dc5200-5f81-41ed-a241-a463bda8be58-additional-service-ca\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.021766 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021678    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-klusterlet-config\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.021766 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.021715    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-ca\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.032060 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.032035    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5ff5888c74-8g62g"]
Jul 08 06:56:21.113662 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.113631    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"]
Jul 08 06:56:21.115620 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.115603    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.122126 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122111    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-klusterlet-config\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.122211 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122148    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-ca\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.122211 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122180    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rc2n\" (UniqueName: \"kubernetes.io/projected/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-kube-api-access-2rc2n\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.122211 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122203    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.122333 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122220    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99fsj\" (UniqueName: \"kubernetes.io/projected/77dc5200-5f81-41ed-a241-a463bda8be58-kube-api-access-99fsj\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.122333 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122236    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-tmp\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.122481 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122462    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6967dbd884-9vwkl\" (UID: \"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.122576 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122496    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-hub\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.122576 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122525    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.122745 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122552    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m24vn\" (UniqueName: \"kubernetes.io/projected/0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec-kube-api-access-m24vn\") pod \"managed-serviceaccount-addon-agent-6967dbd884-9vwkl\" (UID: \"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.122745 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122608    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dc5200-5f81-41ed-a241-a463bda8be58-additional-service-ca\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.122745 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.122653    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-tmp\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.123403 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.123376    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dc5200-5f81-41ed-a241-a463bda8be58-additional-service-ca\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.125155 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.125124    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-ca\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.125309 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.125283    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6967dbd884-9vwkl\" (UID: \"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.125398 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.125354    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-klusterlet-config\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.125551 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.125536    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.125801 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.125779    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-hub\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.125975 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.125960    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77dc5200-5f81-41ed-a241-a463bda8be58-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.142272 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.142243    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m24vn\" (UniqueName: \"kubernetes.io/projected/0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec-kube-api-access-m24vn\") pod \"managed-serviceaccount-addon-agent-6967dbd884-9vwkl\" (UID: \"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.142415 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.142338    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99fsj\" (UniqueName: \"kubernetes.io/projected/77dc5200-5f81-41ed-a241-a463bda8be58-kube-api-access-99fsj\") pod \"cluster-proxy-proxy-agent-565cc4987d-6rcnf\" (UID: \"77dc5200-5f81-41ed-a241-a463bda8be58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.142477 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.142444    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rc2n\" (UniqueName: \"kubernetes.io/projected/b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6-kube-api-access-2rc2n\") pod \"klusterlet-addon-workmgr-5759857b5-q27kz\" (UID: \"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.145889 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.145864    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"]
Jul 08 06:56:21.223334 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223300    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24b0353c-e521-4edc-815e-ea9136aa6541-registry-certificates\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223494 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24b0353c-e521-4edc-815e-ea9136aa6541-image-registry-private-configuration\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223494 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223399    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-bound-sa-token\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223494 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223437    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24b0353c-e521-4edc-815e-ea9136aa6541-trusted-ca\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223494 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223477    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24b0353c-e521-4edc-815e-ea9136aa6541-ca-trust-extracted\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223734 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223512    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24b0353c-e521-4edc-815e-ea9136aa6541-installation-pull-secrets\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223734 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-registry-tls\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.223734 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.223645    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sggkk\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-kube-api-access-sggkk\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.316614 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.316518    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"
Jul 08 06:56:21.323259 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.323238    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"
Jul 08 06:56:21.324575 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324543    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24b0353c-e521-4edc-815e-ea9136aa6541-installation-pull-secrets\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.324685 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324613    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-registry-tls\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.324685 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324675    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sggkk\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-kube-api-access-sggkk\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.324798 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324724    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24b0353c-e521-4edc-815e-ea9136aa6541-registry-certificates\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.324798 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324769    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24b0353c-e521-4edc-815e-ea9136aa6541-image-registry-private-configuration\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.324907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324801    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-bound-sa-token\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.324966 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24b0353c-e521-4edc-815e-ea9136aa6541-trusted-ca\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.325018 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.324988    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24b0353c-e521-4edc-815e-ea9136aa6541-ca-trust-extracted\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.325837 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.325769    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24b0353c-e521-4edc-815e-ea9136aa6541-ca-trust-extracted\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.325999 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.325885    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24b0353c-e521-4edc-815e-ea9136aa6541-trusted-ca\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.326113 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.326091    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24b0353c-e521-4edc-815e-ea9136aa6541-registry-certificates\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.327494 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.327470    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24b0353c-e521-4edc-815e-ea9136aa6541-image-registry-private-configuration\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.327824 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.327783    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-registry-tls\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.327824 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.327800    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24b0353c-e521-4edc-815e-ea9136aa6541-installation-pull-secrets\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.333744 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.333723    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-bound-sa-token\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.334114 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.334091    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sggkk\" (UniqueName: \"kubernetes.io/projected/24b0353c-e521-4edc-815e-ea9136aa6541-kube-api-access-sggkk\") pod \"image-registry-7c9bb86f5c-2gs7f\" (UID: \"24b0353c-e521-4edc-815e-ea9136aa6541\") " pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.342814 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.342788    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:21.424394 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.424347    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:21.461542 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.461509    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl"]
Jul 08 06:56:21.467359 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:21.467308    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b45dcf1_ade0_4c8f_8c71_a87e28bc1eec.slice/crio-77b7db432c4fb3473bc1c0fdd8eaea3885ef89d06151195bef8821400791d835 WatchSource:0}: Error finding container 77b7db432c4fb3473bc1c0fdd8eaea3885ef89d06151195bef8821400791d835: Status 404 returned error can't find the container with id 77b7db432c4fb3473bc1c0fdd8eaea3885ef89d06151195bef8821400791d835
Jul 08 06:56:21.484924 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.484862    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf"]
Jul 08 06:56:21.489421 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:21.489396    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77dc5200_5f81_41ed_a241_a463bda8be58.slice/crio-a3c12d2302a5ae21e6acacd3e22103b38f36dc81c76a54758e948bde08a3528c WatchSource:0}: Error finding container a3c12d2302a5ae21e6acacd3e22103b38f36dc81c76a54758e948bde08a3528c: Status 404 returned error can't find the container with id a3c12d2302a5ae21e6acacd3e22103b38f36dc81c76a54758e948bde08a3528c
Jul 08 06:56:21.498115 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.498092    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"]
Jul 08 06:56:21.501474 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:21.501446    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62f8f71_9dd1_42ed_96a8_d59e8ac25fb6.slice/crio-013c7ec29eb03f0cde9e9fd7db4f981b014b8333c10b8f24a746d2966f627749 WatchSource:0}: Error finding container 013c7ec29eb03f0cde9e9fd7db4f981b014b8333c10b8f24a746d2966f627749: Status 404 returned error can't find the container with id 013c7ec29eb03f0cde9e9fd7db4f981b014b8333c10b8f24a746d2966f627749
Jul 08 06:56:21.562846 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:21.562813    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"]
Jul 08 06:56:21.565513 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:21.565487    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b0353c_e521_4edc_815e_ea9136aa6541.slice/crio-af0d94d943abdd42647a63c7044cdb72fcd068e668b40b04f47bb0df7f642b87 WatchSource:0}: Error finding container af0d94d943abdd42647a63c7044cdb72fcd068e668b40b04f47bb0df7f642b87: Status 404 returned error can't find the container with id af0d94d943abdd42647a63c7044cdb72fcd068e668b40b04f47bb0df7f642b87
Jul 08 06:56:22.211398 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.211325    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf" event={"ID":"77dc5200-5f81-41ed-a241-a463bda8be58","Type":"ContainerStarted","Data":"a3c12d2302a5ae21e6acacd3e22103b38f36dc81c76a54758e948bde08a3528c"}
Jul 08 06:56:22.214818 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.214792    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl" event={"ID":"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec","Type":"ContainerStarted","Data":"77b7db432c4fb3473bc1c0fdd8eaea3885ef89d06151195bef8821400791d835"}
Jul 08 06:56:22.216763 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.216735    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f" event={"ID":"24b0353c-e521-4edc-815e-ea9136aa6541","Type":"ContainerStarted","Data":"93bbed63c89ee49b434a3f28b2b1ef187545b6b34fb401c914e63443ed163579"}
Jul 08 06:56:22.216862 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.216784    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f" event={"ID":"24b0353c-e521-4edc-815e-ea9136aa6541","Type":"ContainerStarted","Data":"af0d94d943abdd42647a63c7044cdb72fcd068e668b40b04f47bb0df7f642b87"}
Jul 08 06:56:22.217527 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.217506    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:22.221484 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.221444    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz" event={"ID":"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6","Type":"ContainerStarted","Data":"013c7ec29eb03f0cde9e9fd7db4f981b014b8333c10b8f24a746d2966f627749"}
Jul 08 06:56:22.246159 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:22.245128    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f" podStartSLOduration=1.245109251 podStartE2EDuration="1.245109251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:22.244402201 +0000 UTC m=+58.031238554" watchObservedRunningTime="2026-07-08 06:56:22.245109251 +0000 UTC m=+58.031945565"
Jul 08 06:56:26.122686 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:26.122660    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4jqd"
Jul 08 06:56:28.198900 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.198871    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7lqgk"
Jul 08 06:56:28.251768 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.251718    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz" event={"ID":"b62f8f71-9dd1-42ed-96a8-d59e8ac25fb6","Type":"ContainerStarted","Data":"0ee97e767394bb2fe1d1a8dd7dc3e7d769461b83d23e9cff00b9133e4e04e4d6"}
Jul 08 06:56:28.251952 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.251921    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:28.253668 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.253636    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz"
Jul 08 06:56:28.254353 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.254325    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf" event={"ID":"77dc5200-5f81-41ed-a241-a463bda8be58","Type":"ContainerStarted","Data":"d425c4a865a70518ce60a45996b10b383ee56ba68605c78b37044d544e464692"}
Jul 08 06:56:28.254466 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.254361    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf" event={"ID":"77dc5200-5f81-41ed-a241-a463bda8be58","Type":"ContainerStarted","Data":"3e7afa68ba9081b07e987c30788ad3a921278a015429bcbaef72d194615b047e"}
Jul 08 06:56:28.254466 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.254387    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf" event={"ID":"77dc5200-5f81-41ed-a241-a463bda8be58","Type":"ContainerStarted","Data":"4d8dfe3aba3bb9ef8cd372ecf0ef06edfe8183e35bf4767474b42aafc817276e"}
Jul 08 06:56:28.255907 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.255887    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl" event={"ID":"0b45dcf1-ade0-4c8f-8c71-a87e28bc1eec","Type":"ContainerStarted","Data":"e6a950a9e10b4c05657c346f13af2424dc71b177a164020a33e347d60d95b65c"}
Jul 08 06:56:28.268211 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.268162    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5759857b5-q27kz" podStartSLOduration=2.517591834 podStartE2EDuration="8.268143606s" totalImagesPullingTime="5.750551772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:28.2678275 +0000 UTC m=+64.054663816" watchObservedRunningTime="2026-07-08 06:56:28.268143606 +0000 UTC m=+64.054979923"
Jul 08 06:56:28.301906 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.301856    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-565cc4987d-6rcnf" podStartSLOduration=2.538122092 podStartE2EDuration="8.301838379s" totalImagesPullingTime="5.763716287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:28.285335448 +0000 UTC m=+64.072171792" watchObservedRunningTime="2026-07-08 06:56:28.301838379 +0000 UTC m=+64.088674697"
Jul 08 06:56:28.315709 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:28.315664    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6967dbd884-9vwkl" podStartSLOduration=2.553629319 podStartE2EDuration="8.315648597s" totalImagesPullingTime="5.762019278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:28.315482908 +0000 UTC m=+64.102319227" watchObservedRunningTime="2026-07-08 06:56:28.315648597 +0000 UTC m=+64.102484912"
Jul 08 06:56:30.504200 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.504167    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:56:30.504200 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.504201    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:56:30.507252 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.507236    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 06:56:30.508607 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.508590    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 06:56:30.516963 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.516945    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fcd81370-ee72-4e93-9089-44790495c67b-original-pull-secret\") pod \"global-pull-secret-syncer-xpq58\" (UID: \"fcd81370-ee72-4e93-9089-44790495c67b\") " pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:56:30.517073 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.517058    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6de506fd-0787-407c-97e3-349973298b4f-metrics-certs\") pod \"network-metrics-daemon-mlrcx\" (UID: \"6de506fd-0787-407c-97e3-349973298b4f\") " pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:56:30.557861 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.557832    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jfr79\""
Jul 08 06:56:30.561173 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.561160    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpq58"
Jul 08 06:56:30.566096 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.566081    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlrcx"
Jul 08 06:56:30.605535 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.605498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:56:30.608784 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.608763    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 06:56:30.619477 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.619446    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 06:56:30.630555 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.630520    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-699rd\" (UniqueName: \"kubernetes.io/projected/484faa4e-d43a-4e7a-9f26-8373ae6cafde-kube-api-access-699rd\") pod \"network-check-target-j48lr\" (UID: \"484faa4e-d43a-4e7a-9f26-8373ae6cafde\") " pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:56:30.661418 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.661393    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-97jj5\""
Jul 08 06:56:30.668826 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.668795    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:56:30.692498 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.692472    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xpq58"]
Jul 08 06:56:30.696117 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:30.696089    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd81370_ee72_4e93_9089_44790495c67b.slice/crio-ad111b1aa659707e6eda1aa9ae5df9c45795d9dffc33039b30ced06e028a6018 WatchSource:0}: Error finding container ad111b1aa659707e6eda1aa9ae5df9c45795d9dffc33039b30ced06e028a6018: Status 404 returned error can't find the container with id ad111b1aa659707e6eda1aa9ae5df9c45795d9dffc33039b30ced06e028a6018
Jul 08 06:56:30.713998 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.713975    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlrcx"]
Jul 08 06:56:30.716237 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:30.716210    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de506fd_0787_407c_97e3_349973298b4f.slice/crio-d9d93d606ffbd87016f388710e7da04ad2a7f743744d928bf1c519a846b77caa WatchSource:0}: Error finding container d9d93d606ffbd87016f388710e7da04ad2a7f743744d928bf1c519a846b77caa: Status 404 returned error can't find the container with id d9d93d606ffbd87016f388710e7da04ad2a7f743744d928bf1c519a846b77caa
Jul 08 06:56:30.797942 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:30.797874    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j48lr"]
Jul 08 06:56:30.800574 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:30.800540    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484faa4e_d43a_4e7a_9f26_8373ae6cafde.slice/crio-5a1e299001aa92e974ea92f65a2cb63b9c3f325fab20096b77a7241a2954be11 WatchSource:0}: Error finding container 5a1e299001aa92e974ea92f65a2cb63b9c3f325fab20096b77a7241a2954be11: Status 404 returned error can't find the container with id 5a1e299001aa92e974ea92f65a2cb63b9c3f325fab20096b77a7241a2954be11
Jul 08 06:56:31.038296 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.038264    2585 patch_prober.go:28] interesting pod/image-registry-5ff5888c74-8g62g container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 06:56:31.038472 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.038314    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" podUID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 06:56:31.267858 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.267818    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlrcx" event={"ID":"6de506fd-0787-407c-97e3-349973298b4f","Type":"ContainerStarted","Data":"d9d93d606ffbd87016f388710e7da04ad2a7f743744d928bf1c519a846b77caa"}
Jul 08 06:56:31.270057 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.270022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xpq58" event={"ID":"fcd81370-ee72-4e93-9089-44790495c67b","Type":"ContainerStarted","Data":"ad111b1aa659707e6eda1aa9ae5df9c45795d9dffc33039b30ced06e028a6018"}
Jul 08 06:56:31.272066 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.272037    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j48lr" event={"ID":"484faa4e-d43a-4e7a-9f26-8373ae6cafde","Type":"ContainerStarted","Data":"5a1e299001aa92e974ea92f65a2cb63b9c3f325fab20096b77a7241a2954be11"}
Jul 08 06:56:31.513627 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.513580    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:31.516550 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.516486    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/737866f6-e412-4126-916b-ff22685fc6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcfnd\" (UID: \"737866f6-e412-4126-916b-ff22685fc6b1\") " pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:31.691895 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.691867    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f5bjk\""
Jul 08 06:56:31.698931 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.698904    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fcfnd"
Jul 08 06:56:31.917255 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:31.917087    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fcfnd"]
Jul 08 06:56:32.072054 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:32.071969    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737866f6_e412_4126_916b_ff22685fc6b1.slice/crio-6576c0583a0d619beb57d8acfda181c8546fbb8474b2f5d94c16c2fda46357cf WatchSource:0}: Error finding container 6576c0583a0d619beb57d8acfda181c8546fbb8474b2f5d94c16c2fda46357cf: Status 404 returned error can't find the container with id 6576c0583a0d619beb57d8acfda181c8546fbb8474b2f5d94c16c2fda46357cf
Jul 08 06:56:32.284588 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:32.284448    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlrcx" event={"ID":"6de506fd-0787-407c-97e3-349973298b4f","Type":"ContainerStarted","Data":"651ebf79d5abd030e07bf8f47d066e7f9b3a157bbbf0cab3df2996bcb04f8398"}
Jul 08 06:56:32.286877 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:32.286809    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcfnd" event={"ID":"737866f6-e412-4126-916b-ff22685fc6b1","Type":"ContainerStarted","Data":"6576c0583a0d619beb57d8acfda181c8546fbb8474b2f5d94c16c2fda46357cf"}
Jul 08 06:56:33.295296 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:33.295257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlrcx" event={"ID":"6de506fd-0787-407c-97e3-349973298b4f","Type":"ContainerStarted","Data":"51e0dcfbc098bdad3ebd0b434a510913a8930cd6ed2923b4a3c212923e1b2c9f"}
Jul 08 06:56:33.314266 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:33.313979    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mlrcx" podStartSLOduration=66.930956618 podStartE2EDuration="1m8.313956698s" totalImagesPullingTime="1.38300008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:33.311704641 +0000 UTC m=+69.098540958" watchObservedRunningTime="2026-07-08 06:56:33.313956698 +0000 UTC m=+69.100793015"
Jul 08 06:56:36.779953 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.779918    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4gkjf"]
Jul 08 06:56:36.796917 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.796889    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.801912 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.800725    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 06:56:36.801912 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.800966    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 06:56:36.801912 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.801084    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 06:56:36.801912 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.801370    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7bnqw\""
Jul 08 06:56:36.801912 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.801619    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 06:56:36.802289 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.802257    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 06:56:36.802734 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.802541    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 06:56:36.863393 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863344    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-textfile\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863581 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863402    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-sys\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863581 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863436    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-wtmp\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863581 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-tls\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863581 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863522    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-root\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863581 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863551    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znsnz\" (UniqueName: \"kubernetes.io/projected/01e62250-3ed2-411e-b681-4fc78f9728f3-kube-api-access-znsnz\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863856 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863606    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863856 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863635    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01e62250-3ed2-411e-b681-4fc78f9728f3-metrics-client-ca\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.863856 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.863653    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964173 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964130    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01e62250-3ed2-411e-b681-4fc78f9728f3-metrics-client-ca\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964351 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964181    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964351 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964221    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-textfile\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964351 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964260    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-sys\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964351 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964291    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-wtmp\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964351 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964325    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-tls\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964612 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964367    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-root\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964612 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964403    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znsnz\" (UniqueName: \"kubernetes.io/projected/01e62250-3ed2-411e-b681-4fc78f9728f3-kube-api-access-znsnz\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964612 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964447    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.964754 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.964734    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-sys\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.965389 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:36.964918    2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 06:56:36.965389 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:36.964986    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-tls podName:01e62250-3ed2-411e-b681-4fc78f9728f3 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:37.464962939 +0000 UTC m=+73.251799238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-tls") pod "node-exporter-4gkjf" (UID: "01e62250-3ed2-411e-b681-4fc78f9728f3") : secret "node-exporter-tls" not found
Jul 08 06:56:36.965389 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.965114    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-wtmp\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.965389 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.965122    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/01e62250-3ed2-411e-b681-4fc78f9728f3-root\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.965389 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.965310    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-textfile\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.966115 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.965800    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.966115 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.966076    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01e62250-3ed2-411e-b681-4fc78f9728f3-metrics-client-ca\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.967504 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.967476    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:36.981960 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:36.981935    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znsnz\" (UniqueName: \"kubernetes.io/projected/01e62250-3ed2-411e-b681-4fc78f9728f3-kube-api-access-znsnz\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:37.468810 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:37.468772    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-tls\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:37.471553 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:37.471531    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/01e62250-3ed2-411e-b681-4fc78f9728f3-node-exporter-tls\") pod \"node-exporter-4gkjf\" (UID: \"01e62250-3ed2-411e-b681-4fc78f9728f3\") " pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:37.709312 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:37.709274    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4gkjf"
Jul 08 06:56:38.294068 ip-10-0-142-155 kubenswrapper[2585]: W0708 06:56:38.294039    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e62250_3ed2_411e_b681_4fc78f9728f3.slice/crio-b5521fbecf27689ef56cc1db599109941c87832bf11dfd86cb5f8cbeb643a02a WatchSource:0}: Error finding container b5521fbecf27689ef56cc1db599109941c87832bf11dfd86cb5f8cbeb643a02a: Status 404 returned error can't find the container with id b5521fbecf27689ef56cc1db599109941c87832bf11dfd86cb5f8cbeb643a02a
Jul 08 06:56:38.314859 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:38.314828    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4gkjf" event={"ID":"01e62250-3ed2-411e-b681-4fc78f9728f3","Type":"ContainerStarted","Data":"b5521fbecf27689ef56cc1db599109941c87832bf11dfd86cb5f8cbeb643a02a"}
Jul 08 06:56:39.320431 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:39.320400    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xpq58" event={"ID":"fcd81370-ee72-4e93-9089-44790495c67b","Type":"ContainerStarted","Data":"695a4e2997b590def60ddb16b25472ca7fc94daaa969575d3228d112e050913f"}
Jul 08 06:56:39.321886 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:39.321869    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j48lr" event={"ID":"484faa4e-d43a-4e7a-9f26-8373ae6cafde","Type":"ContainerStarted","Data":"7c6759b422128316040f457cc459f32e63c736ec34c54b36e64936838ffa5098"}
Jul 08 06:56:39.321982 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:39.321965    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 06:56:39.323298 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:39.323252    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcfnd" event={"ID":"737866f6-e412-4126-916b-ff22685fc6b1","Type":"ContainerStarted","Data":"bd68f20f80f1beceb72e4c9f28c86b643c14e467e492e749e075bf4315ae84c7"}
Jul 08 06:56:39.336000 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:39.335955    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xpq58" podStartSLOduration=66.744216082 podStartE2EDuration="1m14.335937955s" totalImagesPullingTime="7.591721873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:39.335637961 +0000 UTC m=+75.122474292" watchObservedRunningTime="2026-07-08 06:56:39.335937955 +0000 UTC m=+75.122774272"
Jul 08 06:56:39.350341 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:39.350303    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j48lr" podStartSLOduration=66.862872757 podStartE2EDuration="1m14.350291924s" totalImagesPullingTime="7.487419167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:39.350149749 +0000 UTC m=+75.136986062" watchObservedRunningTime="2026-07-08 06:56:39.350291924 +0000 UTC m=+75.137128222"
Jul 08 06:56:40.330300 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:40.330257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcfnd" event={"ID":"737866f6-e412-4126-916b-ff22685fc6b1","Type":"ContainerStarted","Data":"c2abb26a281da0d8f8f2fdcb5da36e2448675ed6529de7a2d665f19d546c49f4"}
Jul 08 06:56:40.332137 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:40.332111    2585 generic.go:356] "Generic (PLEG): container finished" podID="01e62250-3ed2-411e-b681-4fc78f9728f3" containerID="baaea5a3b8bfc0590ae33b4cf3557cd3e8595e2a67816724fd9440cc66a952c9" exitCode=0
Jul 08 06:56:40.332269 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:40.332202    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4gkjf" event={"ID":"01e62250-3ed2-411e-b681-4fc78f9728f3","Type":"ContainerDied","Data":"baaea5a3b8bfc0590ae33b4cf3557cd3e8595e2a67816724fd9440cc66a952c9"}
Jul 08 06:56:41.038150 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:41.038124    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:41.339041 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:41.338967    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4gkjf" event={"ID":"01e62250-3ed2-411e-b681-4fc78f9728f3","Type":"ContainerStarted","Data":"95915a551c8e9d24964cff6552461d78c65c69078ffead9a95c026e68f1c7c90"}
Jul 08 06:56:41.339041 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:41.339023    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4gkjf" event={"ID":"01e62250-3ed2-411e-b681-4fc78f9728f3","Type":"ContainerStarted","Data":"f290eda6b24e7cf4b77ad077a65ad97c76e5cfa73a59aaf6aaad30734952c1b6"}
Jul 08 06:56:41.364534 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:41.364474    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4gkjf" podStartSLOduration=4.147314872 podStartE2EDuration="5.364451161s" totalImagesPullingTime="1.151998643s" totalInitContainerRuntime="65.137646ms" isStatefulPod=true podCreationTimestamp="2026-07-08 06:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:41.364027345 +0000 UTC m=+77.150863712" watchObservedRunningTime="2026-07-08 06:56:41.364451161 +0000 UTC m=+77.151287479"
Jul 08 06:56:42.344864 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:42.344777    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcfnd" event={"ID":"737866f6-e412-4126-916b-ff22685fc6b1","Type":"ContainerStarted","Data":"eb07d5b8884b2f3f33a211ffa263f4294cdae467983da29ee6e6b9789ebc9cfc"}
Jul 08 06:56:42.365778 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:42.365726    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fcfnd" podStartSLOduration=33.571970981 podStartE2EDuration="43.365710899s" totalImagesPullingTime="9.793739918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:42.363461497 +0000 UTC m=+78.150297812" watchObservedRunningTime="2026-07-08 06:56:42.365710899 +0000 UTC m=+78.152547217"
Jul 08 06:56:44.237598 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:44.237549    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c9bb86f5c-2gs7f"
Jul 08 06:56:46.051494 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.051458    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" podUID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" containerName="registry" containerID="cri-o://35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d" gracePeriod=30
Jul 08 06:56:46.292469 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.292446    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:46.361919 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.361832    2585 generic.go:356] "Generic (PLEG): container finished" podID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" containerID="35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d" exitCode=0
Jul 08 06:56:46.361919 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.361896    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g"
Jul 08 06:56:46.362094 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.361910    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" event={"ID":"1d3fd240-61f7-4cb8-9fea-407f382f6a1b","Type":"ContainerDied","Data":"35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d"}
Jul 08 06:56:46.362094 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.361982    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ff5888c74-8g62g" event={"ID":"1d3fd240-61f7-4cb8-9fea-407f382f6a1b","Type":"ContainerDied","Data":"3f1001c7f2c08419d35ca99ae6e39693f4a9498018bec125d84b36ae720bb4e5"}
Jul 08 06:56:46.362094 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.362012    2585 scope.go:117] "RemoveContainer" containerID="35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d"
Jul 08 06:56:46.369800 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.369779    2585 scope.go:117] "RemoveContainer" containerID="35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d"
Jul 08 06:56:46.370092 ip-10-0-142-155 kubenswrapper[2585]: E0708 06:56:46.370070    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d\": container with ID starting with 35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d not found: ID does not exist" containerID="35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d"
Jul 08 06:56:46.370178 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.370099    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d"} err="failed to get container status \"35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d\": rpc error: code = NotFound desc = could not find container \"35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d\": container with ID starting with 35629ba63b5dffa5d61d0eb124d54ac466f81b2b03d55b6d51af996e6a243f8d not found: ID does not exist"
Jul 08 06:56:46.444944 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.444912    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-image-registry-private-configuration\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445099 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.444970    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-bound-sa-token\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445099 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.444993    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-installation-pull-secrets\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445207 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445131    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-certificates\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445207 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445187    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-trusted-ca\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445313 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445222    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445313 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445255    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-ca-trust-extracted\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445313 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445290    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn46c\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-kube-api-access-mn46c\") pod \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\" (UID: \"1d3fd240-61f7-4cb8-9fea-407f382f6a1b\") "
Jul 08 06:56:46.445609 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445525    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:46.445830 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.445727    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:46.447487 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.447462    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:46.447623 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.447581    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:56:46.447768 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.447741    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:56:46.447855 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.447779    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:46.447898 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.447858    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-kube-api-access-mn46c" (OuterVolumeSpecName: "kube-api-access-mn46c") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "kube-api-access-mn46c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:46.453531 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.453507    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1d3fd240-61f7-4cb8-9fea-407f382f6a1b" (UID: "1d3fd240-61f7-4cb8-9fea-407f382f6a1b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 06:56:46.546391 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546349    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-bound-sa-token\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546391 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546386    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-installation-pull-secrets\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546391 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546397    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-certificates\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546391 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546405    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-trusted-ca\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546677 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546414    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-registry-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546677 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546421    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-ca-trust-extracted\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546677 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546429    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn46c\" (UniqueName: \"kubernetes.io/projected/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-kube-api-access-mn46c\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.546677 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.546439    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d3fd240-61f7-4cb8-9fea-407f382f6a1b-image-registry-private-configuration\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 06:56:46.684028 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.684003    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5ff5888c74-8g62g"]
Jul 08 06:56:46.686997 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.686973    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5ff5888c74-8g62g"]
Jul 08 06:56:46.949184 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:56:46.949153    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" path="/var/lib/kubelet/pods/1d3fd240-61f7-4cb8-9fea-407f382f6a1b/volumes"
Jul 08 06:57:10.335055 ip-10-0-142-155 kubenswrapper[2585]: I0708 06:57:10.335024    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j48lr"
Jul 08 07:00:24.689318 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:00:24.689294    2585 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 07:01:49.334685 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.334601    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-4w4dx"]
Jul 08 07:01:49.335159 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.334979    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" containerName="registry"
Jul 08 07:01:49.335159 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.334995    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" containerName="registry"
Jul 08 07:01:49.335159 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.335051    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3fd240-61f7-4cb8-9fea-407f382f6a1b" containerName="registry"
Jul 08 07:01:49.337647 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.337632    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.340506 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.340484    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 08 07:01:49.340666 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.340599    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 08 07:01:49.340666 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.340615    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-g8j9j\""
Jul 08 07:01:49.341740 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.341720    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 08 07:01:49.347543 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.347523    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-4w4dx"]
Jul 08 07:01:49.504484 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.504444    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0d316631-a099-45f4-9741-2888ee314ea5-data\") pod \"seaweedfs-64568bcd49-4w4dx\" (UID: \"0d316631-a099-45f4-9741-2888ee314ea5\") " pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.504686 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.504496    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsls7\" (UniqueName: \"kubernetes.io/projected/0d316631-a099-45f4-9741-2888ee314ea5-kube-api-access-qsls7\") pod \"seaweedfs-64568bcd49-4w4dx\" (UID: \"0d316631-a099-45f4-9741-2888ee314ea5\") " pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.605007 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.604902    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0d316631-a099-45f4-9741-2888ee314ea5-data\") pod \"seaweedfs-64568bcd49-4w4dx\" (UID: \"0d316631-a099-45f4-9741-2888ee314ea5\") " pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.605007 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.604960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsls7\" (UniqueName: \"kubernetes.io/projected/0d316631-a099-45f4-9741-2888ee314ea5-kube-api-access-qsls7\") pod \"seaweedfs-64568bcd49-4w4dx\" (UID: \"0d316631-a099-45f4-9741-2888ee314ea5\") " pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.605373 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.605349    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0d316631-a099-45f4-9741-2888ee314ea5-data\") pod \"seaweedfs-64568bcd49-4w4dx\" (UID: \"0d316631-a099-45f4-9741-2888ee314ea5\") " pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.614901 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.614869    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsls7\" (UniqueName: \"kubernetes.io/projected/0d316631-a099-45f4-9741-2888ee314ea5-kube-api-access-qsls7\") pod \"seaweedfs-64568bcd49-4w4dx\" (UID: \"0d316631-a099-45f4-9741-2888ee314ea5\") " pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.646691 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.646666    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:49.772980 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.772957    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-4w4dx"]
Jul 08 07:01:49.775310 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:01:49.775283    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d316631_a099_45f4_9741_2888ee314ea5.slice/crio-0fbbde46d1fa6226f68523d8ec86e69ae62cef8ec55840b1f5afa29914f076cb WatchSource:0}: Error finding container 0fbbde46d1fa6226f68523d8ec86e69ae62cef8ec55840b1f5afa29914f076cb: Status 404 returned error can't find the container with id 0fbbde46d1fa6226f68523d8ec86e69ae62cef8ec55840b1f5afa29914f076cb
Jul 08 07:01:49.776640 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:49.776622    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 07:01:50.349989 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:50.349953    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-4w4dx" event={"ID":"0d316631-a099-45f4-9741-2888ee314ea5","Type":"ContainerStarted","Data":"0fbbde46d1fa6226f68523d8ec86e69ae62cef8ec55840b1f5afa29914f076cb"}
Jul 08 07:01:53.364196 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:53.364158    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-4w4dx" event={"ID":"0d316631-a099-45f4-9741-2888ee314ea5","Type":"ContainerStarted","Data":"e939b9c574e007cf21a1188d4814dbca33cab9fcb518fe8bae268d1d0f11a37a"}
Jul 08 07:01:53.364651 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:53.364287    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:01:53.380012 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:53.379955    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-4w4dx" podStartSLOduration=1.257247794 podStartE2EDuration="4.37994013s" totalImagesPullingTime="3.122692336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:01:53.379203183 +0000 UTC m=+389.166039523" watchObservedRunningTime="2026-07-08 07:01:53.37994013 +0000 UTC m=+389.166776446"
Jul 08 07:01:59.369739 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:01:59.369706    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-4w4dx"
Jul 08 07:02:59.400417 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.400337    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-tjfgk"]
Jul 08 07:02:59.403279 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.403263    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:02:59.406359 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.406335    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 08 07:02:59.406475 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.406371    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-pc8rc\""
Jul 08 07:02:59.412761 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.412741    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-tjfgk"]
Jul 08 07:02:59.506155 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.506117    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bcee4-10dc-4620-8f16-83882623f7c9-tls-certs\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:02:59.506155 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.506170    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz6qb\" (UniqueName: \"kubernetes.io/projected/ba5bcee4-10dc-4620-8f16-83882623f7c9-kube-api-access-dz6qb\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:02:59.606728 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.606699    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bcee4-10dc-4620-8f16-83882623f7c9-tls-certs\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:02:59.606905 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.606740    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz6qb\" (UniqueName: \"kubernetes.io/projected/ba5bcee4-10dc-4620-8f16-83882623f7c9-kube-api-access-dz6qb\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:02:59.606905 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:02:59.606859    2585 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 08 07:02:59.606976 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:02:59.606939    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5bcee4-10dc-4620-8f16-83882623f7c9-tls-certs podName:ba5bcee4-10dc-4620-8f16-83882623f7c9 nodeName:}" failed. No retries permitted until 2026-07-08 07:03:00.106916488 +0000 UTC m=+455.893752785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ba5bcee4-10dc-4620-8f16-83882623f7c9-tls-certs") pod "model-serving-api-fd65d7d6-tjfgk" (UID: "ba5bcee4-10dc-4620-8f16-83882623f7c9") : secret "model-serving-api-tls" not found
Jul 08 07:02:59.616074 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:02:59.616049    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz6qb\" (UniqueName: \"kubernetes.io/projected/ba5bcee4-10dc-4620-8f16-83882623f7c9-kube-api-access-dz6qb\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:03:00.112372 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:00.112315    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bcee4-10dc-4620-8f16-83882623f7c9-tls-certs\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:03:00.114676 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:00.114655    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bcee4-10dc-4620-8f16-83882623f7c9-tls-certs\") pod \"model-serving-api-fd65d7d6-tjfgk\" (UID: \"ba5bcee4-10dc-4620-8f16-83882623f7c9\") " pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:03:00.314401 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:00.314359    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:03:00.439733 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:00.439695    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-tjfgk"]
Jul 08 07:03:00.442329 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:03:00.442304    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5bcee4_10dc_4620_8f16_83882623f7c9.slice/crio-68be6d407f46375bec8e2e621d1612399c411056b4ef37cc59b541758395561a WatchSource:0}: Error finding container 68be6d407f46375bec8e2e621d1612399c411056b4ef37cc59b541758395561a: Status 404 returned error can't find the container with id 68be6d407f46375bec8e2e621d1612399c411056b4ef37cc59b541758395561a
Jul 08 07:03:00.587656 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:00.587616    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-tjfgk" event={"ID":"ba5bcee4-10dc-4620-8f16-83882623f7c9","Type":"ContainerStarted","Data":"68be6d407f46375bec8e2e621d1612399c411056b4ef37cc59b541758395561a"}
Jul 08 07:03:03.599843 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:03.599804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-tjfgk" event={"ID":"ba5bcee4-10dc-4620-8f16-83882623f7c9","Type":"ContainerStarted","Data":"5360cf574661557ed17cc8bdb676a76027c6e7199eb3df07892ab286f9a8ebeb"}
Jul 08 07:03:03.600233 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:03.599914    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:03:03.617783 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:03.617725    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-tjfgk" podStartSLOduration=2.299807788 podStartE2EDuration="4.617707617s" totalImagesPullingTime="2.317899829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:03:03.616059978 +0000 UTC m=+459.402896321" watchObservedRunningTime="2026-07-08 07:03:03.617707617 +0000 UTC m=+459.404543942"
Jul 08 07:03:14.607276 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:03:14.607247    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-tjfgk"
Jul 08 07:04:47.901293 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.901220    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"]
Jul 08 07:04:47.904357 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.904342    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:47.907055 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.907035    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5f3f4-kube-rbac-proxy-sar-config\""
Jul 08 07:04:47.907312 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.907289    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5f3f4-predictor-serving-cert\""
Jul 08 07:04:47.907403 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.907308    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 07:04:47.908639 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.908622    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jrfgb\""
Jul 08 07:04:47.908731 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.908677    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 07:04:47.914430 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.914408    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"]
Jul 08 07:04:47.999133 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.999094    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ct6r\" (UniqueName: \"kubernetes.io/projected/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-api-access-8ct6r\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:47.999287 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.999163    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6be0241-77c8-40b7-a8fa-a2838950f015-proxy-tls\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:47.999287 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:47.999189    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.100105 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.100072    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ct6r\" (UniqueName: \"kubernetes.io/projected/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-api-access-8ct6r\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.100279 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.100113    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6be0241-77c8-40b7-a8fa-a2838950f015-proxy-tls\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.100279 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.100143    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.100845 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.100821    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.102633 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.102604    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6be0241-77c8-40b7-a8fa-a2838950f015-proxy-tls\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.108271 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.108247    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ct6r\" (UniqueName: \"kubernetes.io/projected/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-api-access-8ct6r\") pod \"error-404-isvc-5f3f4-predictor-595bd58f95-29vch\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") " pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.216105 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.216072    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:04:48.342251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.342075    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"]
Jul 08 07:04:48.344814 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:04:48.344790    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6be0241_77c8_40b7_a8fa_a2838950f015.slice/crio-58b94f1bb912c54496c61d3f9c10eb4c060555c6fc2d96717c717280c19ec6d6 WatchSource:0}: Error finding container 58b94f1bb912c54496c61d3f9c10eb4c060555c6fc2d96717c717280c19ec6d6: Status 404 returned error can't find the container with id 58b94f1bb912c54496c61d3f9c10eb4c060555c6fc2d96717c717280c19ec6d6
Jul 08 07:04:48.952594 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:04:48.952463    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" event={"ID":"f6be0241-77c8-40b7-a8fa-a2838950f015","Type":"ContainerStarted","Data":"58b94f1bb912c54496c61d3f9c10eb4c060555c6fc2d96717c717280c19ec6d6"}
Jul 08 07:05:04.009551 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:04.009469    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" event={"ID":"f6be0241-77c8-40b7-a8fa-a2838950f015","Type":"ContainerStarted","Data":"a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747"}
Jul 08 07:05:06.018767 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:06.018670    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" event={"ID":"f6be0241-77c8-40b7-a8fa-a2838950f015","Type":"ContainerStarted","Data":"587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6"}
Jul 08 07:05:06.019167 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:06.018918    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:05:06.041106 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:06.041058    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podStartSLOduration=1.676173981 podStartE2EDuration="19.041042437s" totalImagesPullingTime="17.364868456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:04:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:05:06.039295926 +0000 UTC m=+581.826132242" watchObservedRunningTime="2026-07-08 07:05:06.041042437 +0000 UTC m=+581.827878752"
Jul 08 07:05:07.022721 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:07.022692    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:05:07.023942 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:07.023915    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 08 07:05:08.026443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:08.026408    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 08 07:05:13.030581 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:13.030540    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:05:13.031089 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:13.031056    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 08 07:05:23.031361 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:23.031317    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 08 07:05:33.031744 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:33.031704    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 08 07:05:43.030971 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:43.030934    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.17:8080: connect: connection refused"
Jul 08 07:05:53.031784 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:05:53.031708    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:06:37.510191 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.510156    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"]
Jul 08 07:06:37.512163 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.512148    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.514938 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.514918    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-5f3f4-serving-cert\""
Jul 08 07:06:37.514938 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.514928    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-5f3f4-kube-rbac-proxy-sar-config\""
Jul 08 07:06:37.527229 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.527205    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"]
Jul 08 07:06:37.608467 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.608429    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af3b6437-b434-41a7-9197-c1e1460f2b0a-proxy-tls\") pod \"switch-graph-5f3f4-759d88bf97-qthpt\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") " pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.608671 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.608492    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3b6437-b434-41a7-9197-c1e1460f2b0a-openshift-service-ca-bundle\") pod \"switch-graph-5f3f4-759d88bf97-qthpt\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") " pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.708911 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.708878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af3b6437-b434-41a7-9197-c1e1460f2b0a-proxy-tls\") pod \"switch-graph-5f3f4-759d88bf97-qthpt\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") " pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.709098 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.708929    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3b6437-b434-41a7-9197-c1e1460f2b0a-openshift-service-ca-bundle\") pod \"switch-graph-5f3f4-759d88bf97-qthpt\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") " pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.709513 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.709493    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3b6437-b434-41a7-9197-c1e1460f2b0a-openshift-service-ca-bundle\") pod \"switch-graph-5f3f4-759d88bf97-qthpt\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") " pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.711262 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.711242    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af3b6437-b434-41a7-9197-c1e1460f2b0a-proxy-tls\") pod \"switch-graph-5f3f4-759d88bf97-qthpt\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") " pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.827139 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.827044    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:37.953625 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:37.953504    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"]
Jul 08 07:06:37.956281 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:06:37.956256    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3b6437_b434_41a7_9197_c1e1460f2b0a.slice/crio-8c8120566f6e4d7de021fcc371b850cc83d7c35a102d5198a6f582ea1a71cc36 WatchSource:0}: Error finding container 8c8120566f6e4d7de021fcc371b850cc83d7c35a102d5198a6f582ea1a71cc36: Status 404 returned error can't find the container with id 8c8120566f6e4d7de021fcc371b850cc83d7c35a102d5198a6f582ea1a71cc36
Jul 08 07:06:38.323757 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:38.323723    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" event={"ID":"af3b6437-b434-41a7-9197-c1e1460f2b0a","Type":"ContainerStarted","Data":"8c8120566f6e4d7de021fcc371b850cc83d7c35a102d5198a6f582ea1a71cc36"}
Jul 08 07:06:41.336940 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:41.336906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" event={"ID":"af3b6437-b434-41a7-9197-c1e1460f2b0a","Type":"ContainerStarted","Data":"a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5"}
Jul 08 07:06:41.337307 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:41.337031    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:41.352625 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:41.352575    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podStartSLOduration=1.987467004 podStartE2EDuration="4.352544105s" totalImagesPullingTime="2.365077101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:06:41.352545316 +0000 UTC m=+677.139381661" watchObservedRunningTime="2026-07-08 07:06:41.352544105 +0000 UTC m=+677.139380421"
Jul 08 07:06:47.347124 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:47.347093    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:06:47.625208 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:47.625125    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"]
Jul 08 07:06:47.625398 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:47.625352    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" containerID="cri-o://a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5" gracePeriod=30
Jul 08 07:06:48.006640 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.006606    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"]
Jul 08 07:06:48.008980 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.008958    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.011729 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.011699    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-104b4-kube-rbac-proxy-sar-config\""
Jul 08 07:06:48.011867 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.011735    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-104b4-predictor-serving-cert\""
Jul 08 07:06:48.019142 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.019117    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"]
Jul 08 07:06:48.044150 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.044119    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"]
Jul 08 07:06:48.044419 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.044389    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container" containerID="cri-o://a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747" gracePeriod=30
Jul 08 07:06:48.044419 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.044406    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kube-rbac-proxy" containerID="cri-o://587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6" gracePeriod=30
Jul 08 07:06:48.086178 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.086147    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.086323 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.086219    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn2cp\" (UniqueName: \"kubernetes.io/projected/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-api-access-pn2cp\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.086323 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.086258    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.186923 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.186886    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn2cp\" (UniqueName: \"kubernetes.io/projected/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-api-access-pn2cp\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.187106 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.186939    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.187106 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.186975    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.187106 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:06:48.187093    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-104b4-predictor-serving-cert: secret "error-404-isvc-104b4-predictor-serving-cert" not found
Jul 08 07:06:48.187257 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:06:48.187151    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls podName:2a560e9d-2692-4715-a3cc-a6b195a40e2e nodeName:}" failed. No retries permitted until 2026-07-08 07:06:48.687130243 +0000 UTC m=+684.473966552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls") pod "error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" (UID: "2a560e9d-2692-4715-a3cc-a6b195a40e2e") : secret "error-404-isvc-104b4-predictor-serving-cert" not found
Jul 08 07:06:48.187706 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.187684    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.198033 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.198006    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn2cp\" (UniqueName: \"kubernetes.io/projected/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-api-access-pn2cp\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.363581 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.363482    2585 generic.go:356] "Generic (PLEG): container finished" podID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerID="587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6" exitCode=2
Jul 08 07:06:48.363581 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.363542    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" event={"ID":"f6be0241-77c8-40b7-a8fa-a2838950f015","Type":"ContainerDied","Data":"587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6"}
Jul 08 07:06:48.690346 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.690303    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.692772 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.692751    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls\") pod \"error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") " pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:48.921503 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:48.921460    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:49.048691 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:49.048666    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"]
Jul 08 07:06:49.051206 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:06:49.051174    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a560e9d_2692_4715_a3cc_a6b195a40e2e.slice/crio-748e2cb4e11664582621b6aef43756e5fdfc86e4d98d19299abdd73a9d1ece1c WatchSource:0}: Error finding container 748e2cb4e11664582621b6aef43756e5fdfc86e4d98d19299abdd73a9d1ece1c: Status 404 returned error can't find the container with id 748e2cb4e11664582621b6aef43756e5fdfc86e4d98d19299abdd73a9d1ece1c
Jul 08 07:06:49.369132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:49.369048    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" event={"ID":"2a560e9d-2692-4715-a3cc-a6b195a40e2e","Type":"ContainerStarted","Data":"34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7"}
Jul 08 07:06:49.369132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:49.369110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" event={"ID":"2a560e9d-2692-4715-a3cc-a6b195a40e2e","Type":"ContainerStarted","Data":"e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05"}
Jul 08 07:06:49.369503 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:49.369142    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:49.369503 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:49.369162    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" event={"ID":"2a560e9d-2692-4715-a3cc-a6b195a40e2e","Type":"ContainerStarted","Data":"748e2cb4e11664582621b6aef43756e5fdfc86e4d98d19299abdd73a9d1ece1c"}
Jul 08 07:06:49.387650 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:49.387606    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podStartSLOduration=2.387589903 podStartE2EDuration="2.387589903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:06:49.386285136 +0000 UTC m=+685.173121451" watchObservedRunningTime="2026-07-08 07:06:49.387589903 +0000 UTC m=+685.174426214"
Jul 08 07:06:50.373482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:50.373452    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:50.374922 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:50.374894    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 08 07:06:51.184309 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.184287    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:06:51.312911 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.312820    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-rbac-proxy-sar-config\") pod \"f6be0241-77c8-40b7-a8fa-a2838950f015\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") "
Jul 08 07:06:51.312911 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.312866    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ct6r\" (UniqueName: \"kubernetes.io/projected/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-api-access-8ct6r\") pod \"f6be0241-77c8-40b7-a8fa-a2838950f015\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") "
Jul 08 07:06:51.312911 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.312894    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6be0241-77c8-40b7-a8fa-a2838950f015-proxy-tls\") pod \"f6be0241-77c8-40b7-a8fa-a2838950f015\" (UID: \"f6be0241-77c8-40b7-a8fa-a2838950f015\") "
Jul 08 07:06:51.313212 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.313180    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f6be0241-77c8-40b7-a8fa-a2838950f015" (UID: "f6be0241-77c8-40b7-a8fa-a2838950f015"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:06:51.315014 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.314986    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-api-access-8ct6r" (OuterVolumeSpecName: "kube-api-access-8ct6r") pod "f6be0241-77c8-40b7-a8fa-a2838950f015" (UID: "f6be0241-77c8-40b7-a8fa-a2838950f015"). InnerVolumeSpecName "kube-api-access-8ct6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:06:51.315014 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.314999    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6be0241-77c8-40b7-a8fa-a2838950f015-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f6be0241-77c8-40b7-a8fa-a2838950f015" (UID: "f6be0241-77c8-40b7-a8fa-a2838950f015"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:06:51.377643 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.377603    2585 generic.go:356] "Generic (PLEG): container finished" podID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerID="a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747" exitCode=0
Jul 08 07:06:51.378096 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.377658    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"
Jul 08 07:06:51.378096 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.377684    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" event={"ID":"f6be0241-77c8-40b7-a8fa-a2838950f015","Type":"ContainerDied","Data":"a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747"}
Jul 08 07:06:51.378096 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.377759    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch" event={"ID":"f6be0241-77c8-40b7-a8fa-a2838950f015","Type":"ContainerDied","Data":"58b94f1bb912c54496c61d3f9c10eb4c060555c6fc2d96717c717280c19ec6d6"}
Jul 08 07:06:51.378096 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.377806    2585 scope.go:117] "RemoveContainer" containerID="587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6"
Jul 08 07:06:51.378298 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.378176    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 08 07:06:51.385619 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.385597    2585 scope.go:117] "RemoveContainer" containerID="a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747"
Jul 08 07:06:51.392391 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.392372    2585 scope.go:117] "RemoveContainer" containerID="587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6"
Jul 08 07:06:51.392646 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:06:51.392628    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6\": container with ID starting with 587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6 not found: ID does not exist" containerID="587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6"
Jul 08 07:06:51.392694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.392653    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6"} err="failed to get container status \"587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6\": rpc error: code = NotFound desc = could not find container \"587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6\": container with ID starting with 587da74050af7950962c68d527c821e52b73495669468d144d272dd7cf0349f6 not found: ID does not exist"
Jul 08 07:06:51.392694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.392668    2585 scope.go:117] "RemoveContainer" containerID="a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747"
Jul 08 07:06:51.392883 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:06:51.392863    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747\": container with ID starting with a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747 not found: ID does not exist" containerID="a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747"
Jul 08 07:06:51.392925 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.392891    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747"} err="failed to get container status \"a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747\": rpc error: code = NotFound desc = could not find container \"a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747\": container with ID starting with a64722b55242bb5a108b97c8131fabc5d1550fa75d351a5887a21eaf5d291747 not found: ID does not exist"
Jul 08 07:06:51.399244 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.399222    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"]
Jul 08 07:06:51.402494 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.402474    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5f3f4-predictor-595bd58f95-29vch"]
Jul 08 07:06:51.413520 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.413504    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:06:51.413618 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.413522    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ct6r\" (UniqueName: \"kubernetes.io/projected/f6be0241-77c8-40b7-a8fa-a2838950f015-kube-api-access-8ct6r\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:06:51.413618 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:51.413532    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6be0241-77c8-40b7-a8fa-a2838950f015-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:06:52.344838 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:52.344803    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:06:52.949018 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:52.948985    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" path="/var/lib/kubelet/pods/f6be0241-77c8-40b7-a8fa-a2838950f015/volumes"
Jul 08 07:06:56.383097 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:56.383065    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:06:56.383490 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:56.383457    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 08 07:06:57.345767 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:06:57.345729    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:02.345525 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:02.345479    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:02.345918 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:02.345606    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:07:06.383805 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:06.383769    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 08 07:07:07.345145 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.345110    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:07.475981 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.475950    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"]
Jul 08 07:07:07.476345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.476229    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container"
Jul 08 07:07:07.476345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.476239    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container"
Jul 08 07:07:07.476345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.476249    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kube-rbac-proxy"
Jul 08 07:07:07.476345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.476255    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kube-rbac-proxy"
Jul 08 07:07:07.476345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.476297    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kserve-container"
Jul 08 07:07:07.476345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.476304    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be0241-77c8-40b7-a8fa-a2838950f015" containerName="kube-rbac-proxy"
Jul 08 07:07:07.480547 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.480529    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.483126 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.483101    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Jul 08 07:07:07.483239 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.483159    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Jul 08 07:07:07.487809 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.487787    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"]
Jul 08 07:07:07.542306 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.542265    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9c0f57e-492f-49bc-ad1e-57de426dfa62-openshift-service-ca-bundle\") pod \"model-chainer-57956d54d6-gk2dr\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") " pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.542502 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.542374    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9c0f57e-492f-49bc-ad1e-57de426dfa62-proxy-tls\") pod \"model-chainer-57956d54d6-gk2dr\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") " pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.642958 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.642879    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9c0f57e-492f-49bc-ad1e-57de426dfa62-openshift-service-ca-bundle\") pod \"model-chainer-57956d54d6-gk2dr\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") " pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.642958 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.642937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9c0f57e-492f-49bc-ad1e-57de426dfa62-proxy-tls\") pod \"model-chainer-57956d54d6-gk2dr\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") " pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.643527 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.643505    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9c0f57e-492f-49bc-ad1e-57de426dfa62-openshift-service-ca-bundle\") pod \"model-chainer-57956d54d6-gk2dr\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") " pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.645275 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.645250    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9c0f57e-492f-49bc-ad1e-57de426dfa62-proxy-tls\") pod \"model-chainer-57956d54d6-gk2dr\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") " pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.791705 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.791669    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:07.916457 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:07.916382    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"]
Jul 08 07:07:07.919331 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:07:07.919300    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c0f57e_492f_49bc_ad1e_57de426dfa62.slice/crio-256b900b1dcdf8579cc6cc13cce16b1937466439a58765affc8161d545428142 WatchSource:0}: Error finding container 256b900b1dcdf8579cc6cc13cce16b1937466439a58765affc8161d545428142: Status 404 returned error can't find the container with id 256b900b1dcdf8579cc6cc13cce16b1937466439a58765affc8161d545428142
Jul 08 07:07:08.437200 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:08.437163    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" event={"ID":"f9c0f57e-492f-49bc-ad1e-57de426dfa62","Type":"ContainerStarted","Data":"66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365"}
Jul 08 07:07:08.437459 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:08.437224    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" event={"ID":"f9c0f57e-492f-49bc-ad1e-57de426dfa62","Type":"ContainerStarted","Data":"256b900b1dcdf8579cc6cc13cce16b1937466439a58765affc8161d545428142"}
Jul 08 07:07:08.437459 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:08.437266    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:08.453583 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:08.453514    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podStartSLOduration=1.453500168 podStartE2EDuration="1.453500168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:07:08.453288182 +0000 UTC m=+704.240124499" watchObservedRunningTime="2026-07-08 07:07:08.453500168 +0000 UTC m=+704.240336483"
Jul 08 07:07:12.345429 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:12.345394    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:14.447184 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:14.447152    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:16.384499 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:16.384412    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 08 07:07:17.345525 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.345488    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:17.576999 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.576967    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"]
Jul 08 07:07:17.577379 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.577135    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" containerID="cri-o://66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365" gracePeriod=30
Jul 08 07:07:17.840325 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.840285    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"]
Jul 08 07:07:17.844351 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.844335    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:17.847078 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.847049    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1469a-predictor-serving-cert\""
Jul 08 07:07:17.847201 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.847149    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1469a-kube-rbac-proxy-sar-config\""
Jul 08 07:07:17.854237 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.854213    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"]
Jul 08 07:07:17.921641 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.921533    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:17.921641 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.921596    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvqc7\" (UniqueName: \"kubernetes.io/projected/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-api-access-lvqc7\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:17.921835 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:17.921673    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.022778 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.022737    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.022778 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.022778    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvqc7\" (UniqueName: \"kubernetes.io/projected/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-api-access-lvqc7\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.022993 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.022856    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.022993 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:07:18.022896    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-1469a-predictor-serving-cert: secret "error-404-isvc-1469a-predictor-serving-cert" not found
Jul 08 07:07:18.022993 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:07:18.022963    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls podName:c67af24e-21d8-41f8-a84a-149dcc5350df nodeName:}" failed. No retries permitted until 2026-07-08 07:07:18.522941854 +0000 UTC m=+714.309778163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls") pod "error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" (UID: "c67af24e-21d8-41f8-a84a-149dcc5350df") : secret "error-404-isvc-1469a-predictor-serving-cert" not found
Jul 08 07:07:18.023476 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.023457    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.031951 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.031881    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvqc7\" (UniqueName: \"kubernetes.io/projected/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-api-access-lvqc7\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.262055 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.262028    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:07:18.425305 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.425271    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af3b6437-b434-41a7-9197-c1e1460f2b0a-proxy-tls\") pod \"af3b6437-b434-41a7-9197-c1e1460f2b0a\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") "
Jul 08 07:07:18.425305 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.425307    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3b6437-b434-41a7-9197-c1e1460f2b0a-openshift-service-ca-bundle\") pod \"af3b6437-b434-41a7-9197-c1e1460f2b0a\" (UID: \"af3b6437-b434-41a7-9197-c1e1460f2b0a\") "
Jul 08 07:07:18.425746 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.425720    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af3b6437-b434-41a7-9197-c1e1460f2b0a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "af3b6437-b434-41a7-9197-c1e1460f2b0a" (UID: "af3b6437-b434-41a7-9197-c1e1460f2b0a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:07:18.427321 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.427300    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af3b6437-b434-41a7-9197-c1e1460f2b0a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "af3b6437-b434-41a7-9197-c1e1460f2b0a" (UID: "af3b6437-b434-41a7-9197-c1e1460f2b0a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:07:18.474385 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.474298    2585 generic.go:356] "Generic (PLEG): container finished" podID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerID="a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5" exitCode=0
Jul 08 07:07:18.474385 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.474344    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" event={"ID":"af3b6437-b434-41a7-9197-c1e1460f2b0a","Type":"ContainerDied","Data":"a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5"}
Jul 08 07:07:18.474578 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.474385    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt" event={"ID":"af3b6437-b434-41a7-9197-c1e1460f2b0a","Type":"ContainerDied","Data":"8c8120566f6e4d7de021fcc371b850cc83d7c35a102d5198a6f582ea1a71cc36"}
Jul 08 07:07:18.474578 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.474410    2585 scope.go:117] "RemoveContainer" containerID="a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5"
Jul 08 07:07:18.474578 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.474343    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"
Jul 08 07:07:18.482388 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.482286    2585 scope.go:117] "RemoveContainer" containerID="a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5"
Jul 08 07:07:18.482632 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:07:18.482607    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5\": container with ID starting with a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5 not found: ID does not exist" containerID="a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5"
Jul 08 07:07:18.482712 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.482643    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5"} err="failed to get container status \"a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5\": rpc error: code = NotFound desc = could not find container \"a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5\": container with ID starting with a60bb971d46248dff033d2d8e78a437476c42539a8d11979822f14861eeab4d5 not found: ID does not exist"
Jul 08 07:07:18.495927 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.495902    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"]
Jul 08 07:07:18.499787 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.499760    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5f3f4-759d88bf97-qthpt"]
Jul 08 07:07:18.525869 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.525842    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.525958 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.525922    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af3b6437-b434-41a7-9197-c1e1460f2b0a-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:07:18.525958 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.525936    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3b6437-b434-41a7-9197-c1e1460f2b0a-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:07:18.528254 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.528233    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls\") pod \"error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") " pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.755984 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.755896    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:18.888208 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.888185    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"]
Jul 08 07:07:18.890466 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:07:18.890437    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67af24e_21d8_41f8_a84a_149dcc5350df.slice/crio-aaccb57cb059095be62726d0361a4549869e505c3f06e7e92857788c0e2cb895 WatchSource:0}: Error finding container aaccb57cb059095be62726d0361a4549869e505c3f06e7e92857788c0e2cb895: Status 404 returned error can't find the container with id aaccb57cb059095be62726d0361a4549869e505c3f06e7e92857788c0e2cb895
Jul 08 07:07:18.949246 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:18.949222    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" path="/var/lib/kubelet/pods/af3b6437-b434-41a7-9197-c1e1460f2b0a/volumes"
Jul 08 07:07:19.444520 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:19.444482    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:19.480070 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:19.480032    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" event={"ID":"c67af24e-21d8-41f8-a84a-149dcc5350df","Type":"ContainerStarted","Data":"01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f"}
Jul 08 07:07:19.480234 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:19.480139    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:19.480234 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:19.480170    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" event={"ID":"c67af24e-21d8-41f8-a84a-149dcc5350df","Type":"ContainerStarted","Data":"8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97"}
Jul 08 07:07:19.480234 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:19.480199    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" event={"ID":"c67af24e-21d8-41f8-a84a-149dcc5350df","Type":"ContainerStarted","Data":"aaccb57cb059095be62726d0361a4549869e505c3f06e7e92857788c0e2cb895"}
Jul 08 07:07:19.497323 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:19.497271    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podStartSLOduration=2.497258413 podStartE2EDuration="2.497258413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:07:19.496869637 +0000 UTC m=+715.283705956" watchObservedRunningTime="2026-07-08 07:07:19.497258413 +0000 UTC m=+715.284094765"
Jul 08 07:07:20.484457 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:20.484428    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:20.485722 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:20.485691    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 08 07:07:21.488242 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:21.488205    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 08 07:07:24.445288 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:24.445246    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:26.383509 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:26.383468    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 08 07:07:26.492955 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:26.492927    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:07:26.493326 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:26.493303    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 08 07:07:29.445276 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:29.445232    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:29.445659 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:29.445338    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:34.444532 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:34.444490    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:36.383700 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:36.383672    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:07:36.493902 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:36.493863    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 08 07:07:39.445228 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:39.445192    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:44.445046 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:44.445009    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:07:46.494080 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:46.494041    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 08 07:07:48.222808 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.222785    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:48.247287 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.247259    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9c0f57e-492f-49bc-ad1e-57de426dfa62-openshift-service-ca-bundle\") pod \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") "
Jul 08 07:07:48.247435 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.247302    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9c0f57e-492f-49bc-ad1e-57de426dfa62-proxy-tls\") pod \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\" (UID: \"f9c0f57e-492f-49bc-ad1e-57de426dfa62\") "
Jul 08 07:07:48.247643 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.247621    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c0f57e-492f-49bc-ad1e-57de426dfa62-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "f9c0f57e-492f-49bc-ad1e-57de426dfa62" (UID: "f9c0f57e-492f-49bc-ad1e-57de426dfa62"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:07:48.249270 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.249245    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9c0f57e-492f-49bc-ad1e-57de426dfa62-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f9c0f57e-492f-49bc-ad1e-57de426dfa62" (UID: "f9c0f57e-492f-49bc-ad1e-57de426dfa62"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:07:48.348479 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.348396    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9c0f57e-492f-49bc-ad1e-57de426dfa62-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:07:48.348479 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.348426    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9c0f57e-492f-49bc-ad1e-57de426dfa62-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:07:48.581228 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.581186    2585 generic.go:356] "Generic (PLEG): container finished" podID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerID="66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365" exitCode=0
Jul 08 07:07:48.581228 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.581233    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"
Jul 08 07:07:48.581463 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.581267    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" event={"ID":"f9c0f57e-492f-49bc-ad1e-57de426dfa62","Type":"ContainerDied","Data":"66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365"}
Jul 08 07:07:48.581463 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.581326    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr" event={"ID":"f9c0f57e-492f-49bc-ad1e-57de426dfa62","Type":"ContainerDied","Data":"256b900b1dcdf8579cc6cc13cce16b1937466439a58765affc8161d545428142"}
Jul 08 07:07:48.581463 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.581354    2585 scope.go:117] "RemoveContainer" containerID="66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365"
Jul 08 07:07:48.591450 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.591433    2585 scope.go:117] "RemoveContainer" containerID="66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365"
Jul 08 07:07:48.591836 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:07:48.591820    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365\": container with ID starting with 66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365 not found: ID does not exist" containerID="66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365"
Jul 08 07:07:48.591889 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.591846    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365"} err="failed to get container status \"66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365\": rpc error: code = NotFound desc = could not find container \"66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365\": container with ID starting with 66e623e4c4fe9e35e1f3abf4c27f4cd3d772350c35b810cc5c38990a10378365 not found: ID does not exist"
Jul 08 07:07:48.602182 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.602125    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"]
Jul 08 07:07:48.605838 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.605818    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-57956d54d6-gk2dr"]
Jul 08 07:07:48.948843 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:48.948812    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" path="/var/lib/kubelet/pods/f9c0f57e-492f-49bc-ad1e-57de426dfa62/volumes"
Jul 08 07:07:56.494121 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:56.494082    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 08 07:07:57.898216 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898183    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"]
Jul 08 07:07:57.898694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898456    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer"
Jul 08 07:07:57.898694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898466    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer"
Jul 08 07:07:57.898694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898475    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4"
Jul 08 07:07:57.898694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898481    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4"
Jul 08 07:07:57.898694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898536    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c0f57e-492f-49bc-ad1e-57de426dfa62" containerName="model-chainer"
Jul 08 07:07:57.898694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.898543    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3b6437-b434-41a7-9197-c1e1460f2b0a" containerName="switch-graph-5f3f4"
Jul 08 07:07:57.901207 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.901190    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:57.905425 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.905402    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-104b4-serving-cert\""
Jul 08 07:07:57.906135 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.906118    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-104b4-kube-rbac-proxy-sar-config\""
Jul 08 07:07:57.911486 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.911464    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"]
Jul 08 07:07:57.917928 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.917906    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b115ebea-050e-472d-b394-8c5cf46ee772-openshift-service-ca-bundle\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:57.918023 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:57.917947    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.018964 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.018927    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b115ebea-050e-472d-b394-8c5cf46ee772-openshift-service-ca-bundle\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.019355 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.019016    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.019355 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:07:58.019101    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-104b4-serving-cert: secret "switch-graph-104b4-serving-cert" not found
Jul 08 07:07:58.019355 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:07:58.019167    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls podName:b115ebea-050e-472d-b394-8c5cf46ee772 nodeName:}" failed. No retries permitted until 2026-07-08 07:07:58.519144538 +0000 UTC m=+754.305980851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls") pod "switch-graph-104b4-c874cb76f-zv7n6" (UID: "b115ebea-050e-472d-b394-8c5cf46ee772") : secret "switch-graph-104b4-serving-cert" not found
Jul 08 07:07:58.019623 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.019604    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b115ebea-050e-472d-b394-8c5cf46ee772-openshift-service-ca-bundle\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.523199 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.523146    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.525473 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.525447    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls\") pod \"switch-graph-104b4-c874cb76f-zv7n6\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") " pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.813196 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.813097    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:58.941433 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:58.941395    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"]
Jul 08 07:07:58.944307 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:07:58.944273    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb115ebea_050e_472d_b394_8c5cf46ee772.slice/crio-c70947a9ea73e51a44a87b98a7e00ddc85d94dd2a1c82059b007c442ccd37b48 WatchSource:0}: Error finding container c70947a9ea73e51a44a87b98a7e00ddc85d94dd2a1c82059b007c442ccd37b48: Status 404 returned error can't find the container with id c70947a9ea73e51a44a87b98a7e00ddc85d94dd2a1c82059b007c442ccd37b48
Jul 08 07:07:59.621246 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:59.621213    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" event={"ID":"b115ebea-050e-472d-b394-8c5cf46ee772","Type":"ContainerStarted","Data":"0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059"}
Jul 08 07:07:59.621420 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:59.621262    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" event={"ID":"b115ebea-050e-472d-b394-8c5cf46ee772","Type":"ContainerStarted","Data":"c70947a9ea73e51a44a87b98a7e00ddc85d94dd2a1c82059b007c442ccd37b48"}
Jul 08 07:07:59.621420 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:59.621339    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:07:59.637905 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:07:59.637855    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podStartSLOduration=2.637840984 podStartE2EDuration="2.637840984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:07:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:07:59.63681841 +0000 UTC m=+755.423654725" watchObservedRunningTime="2026-07-08 07:07:59.637840984 +0000 UTC m=+755.424677299"
Jul 08 07:08:05.631761 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:05.631733    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:08:06.494676 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:06.494643    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:08:27.748341 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.748310    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"]
Jul 08 07:08:27.751622 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.751577    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:27.754326 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.754306    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-1469a-kube-rbac-proxy-sar-config\""
Jul 08 07:08:27.754326 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.754320    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-1469a-serving-cert\""
Jul 08 07:08:27.759923 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.759903    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"]
Jul 08 07:08:27.841040 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.840995    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d48dba0-8ae4-4962-8405-0860ae24850e-openshift-service-ca-bundle\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:27.841232 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.841066    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:27.942457 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.942420    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:27.942457 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.942463    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d48dba0-8ae4-4962-8405-0860ae24850e-openshift-service-ca-bundle\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:27.942733 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:08:27.942590    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-1469a-serving-cert: secret "sequence-graph-1469a-serving-cert" not found
Jul 08 07:08:27.942733 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:08:27.942673    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls podName:7d48dba0-8ae4-4962-8405-0860ae24850e nodeName:}" failed. No retries permitted until 2026-07-08 07:08:28.442653896 +0000 UTC m=+784.229490205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls") pod "sequence-graph-1469a-d7c587ff8-7j2jt" (UID: "7d48dba0-8ae4-4962-8405-0860ae24850e") : secret "sequence-graph-1469a-serving-cert" not found
Jul 08 07:08:27.943054 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:27.943036    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d48dba0-8ae4-4962-8405-0860ae24850e-openshift-service-ca-bundle\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:28.447022 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:28.446967    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:28.449396 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:28.449367    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls\") pod \"sequence-graph-1469a-d7c587ff8-7j2jt\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") " pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:28.663688 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:28.663650    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:28.796587 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:28.796533    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"]
Jul 08 07:08:28.798953 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:08:28.798925    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d48dba0_8ae4_4962_8405_0860ae24850e.slice/crio-54583b00d1d1b5acbac4714c7bd952f1f44c28cd81100f5bf120a5f41bc30e76 WatchSource:0}: Error finding container 54583b00d1d1b5acbac4714c7bd952f1f44c28cd81100f5bf120a5f41bc30e76: Status 404 returned error can't find the container with id 54583b00d1d1b5acbac4714c7bd952f1f44c28cd81100f5bf120a5f41bc30e76
Jul 08 07:08:29.727787 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:29.727751    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" event={"ID":"7d48dba0-8ae4-4962-8405-0860ae24850e","Type":"ContainerStarted","Data":"5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214"}
Jul 08 07:08:29.727961 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:29.727804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" event={"ID":"7d48dba0-8ae4-4962-8405-0860ae24850e","Type":"ContainerStarted","Data":"54583b00d1d1b5acbac4714c7bd952f1f44c28cd81100f5bf120a5f41bc30e76"}
Jul 08 07:08:29.727961 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:29.727830    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:08:29.743657 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:29.743598    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podStartSLOduration=2.743553512 podStartE2EDuration="2.743553512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:08:29.74316093 +0000 UTC m=+785.529997248" watchObservedRunningTime="2026-07-08 07:08:29.743553512 +0000 UTC m=+785.530389829"
Jul 08 07:08:35.737045 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:08:35.737011    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:16:12.540003 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.539969    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"]
Jul 08 07:16:12.540536 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.540166    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" containerID="cri-o://0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059" gracePeriod=30
Jul 08 07:16:12.784545 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.784506    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"]
Jul 08 07:16:12.787717 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.787701    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:12.790896 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.790841    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4baa0-kube-rbac-proxy-sar-config\""
Jul 08 07:16:12.791479 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.791454    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4baa0-predictor-serving-cert\""
Jul 08 07:16:12.800410 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.800389    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"]
Jul 08 07:16:12.827968 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.827942    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"]
Jul 08 07:16:12.828229 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.828166    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container" containerID="cri-o://e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05" gracePeriod=30
Jul 08 07:16:12.828229 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.828188    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kube-rbac-proxy" containerID="cri-o://34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7" gracePeriod=30
Jul 08 07:16:12.923847 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.923813    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnz56\" (UniqueName: \"kubernetes.io/projected/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-api-access-dnz56\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:12.923847 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.923846    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e698b4c-69c9-4188-8dac-709f47e602a4-proxy-tls\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:12.924085 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:12.923882    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.024991 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.024959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnz56\" (UniqueName: \"kubernetes.io/projected/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-api-access-dnz56\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.024991 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.024997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e698b4c-69c9-4188-8dac-709f47e602a4-proxy-tls\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.025246 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.025081    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.025830 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.025807    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.027403 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.027385    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e698b4c-69c9-4188-8dac-709f47e602a4-proxy-tls\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.034358 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.034335    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnz56\" (UniqueName: \"kubernetes.io/projected/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-api-access-dnz56\") pod \"error-404-isvc-4baa0-predictor-ff5686f95-v7hhj\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") " pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.099295 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.099210    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.230788 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.230763    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"]
Jul 08 07:16:13.233464 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:16:13.233437    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e698b4c_69c9_4188_8dac_709f47e602a4.slice/crio-6dbe3322b3e60cf12ada2ee2d013ce78193ff067747bd50cafa866089d104dd4 WatchSource:0}: Error finding container 6dbe3322b3e60cf12ada2ee2d013ce78193ff067747bd50cafa866089d104dd4: Status 404 returned error can't find the container with id 6dbe3322b3e60cf12ada2ee2d013ce78193ff067747bd50cafa866089d104dd4
Jul 08 07:16:13.370461 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.370355    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" event={"ID":"5e698b4c-69c9-4188-8dac-709f47e602a4","Type":"ContainerStarted","Data":"19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb"}
Jul 08 07:16:13.370461 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.370436    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.370695 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.370466    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" event={"ID":"5e698b4c-69c9-4188-8dac-709f47e602a4","Type":"ContainerStarted","Data":"3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5"}
Jul 08 07:16:13.370695 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.370496    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:13.370695 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.370521    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" event={"ID":"5e698b4c-69c9-4188-8dac-709f47e602a4","Type":"ContainerStarted","Data":"6dbe3322b3e60cf12ada2ee2d013ce78193ff067747bd50cafa866089d104dd4"}
Jul 08 07:16:13.372010 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.371982    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 08 07:16:13.375121 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.372507    2585 generic.go:356] "Generic (PLEG): container finished" podID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerID="34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7" exitCode=2
Jul 08 07:16:13.375121 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.372581    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" event={"ID":"2a560e9d-2692-4715-a3cc-a6b195a40e2e","Type":"ContainerDied","Data":"34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7"}
Jul 08 07:16:13.388714 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:13.388672    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podStartSLOduration=1.388656097 podStartE2EDuration="1.388656097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:16:13.387921228 +0000 UTC m=+1249.174757564" watchObservedRunningTime="2026-07-08 07:16:13.388656097 +0000 UTC m=+1249.175492415"
Jul 08 07:16:14.375824 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:14.375785    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 08 07:16:15.631085 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:15.631042    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:15.976621 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:15.976597    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:16:16.052827 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.052788    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-rbac-proxy-sar-config\") pod \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") "
Jul 08 07:16:16.052996 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.052852    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls\") pod \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") "
Jul 08 07:16:16.052996 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.052938    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn2cp\" (UniqueName: \"kubernetes.io/projected/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-api-access-pn2cp\") pod \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\" (UID: \"2a560e9d-2692-4715-a3cc-a6b195a40e2e\") "
Jul 08 07:16:16.053231 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.053202    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2a560e9d-2692-4715-a3cc-a6b195a40e2e" (UID: "2a560e9d-2692-4715-a3cc-a6b195a40e2e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:16:16.054958 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.054931    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2a560e9d-2692-4715-a3cc-a6b195a40e2e" (UID: "2a560e9d-2692-4715-a3cc-a6b195a40e2e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:16:16.055046 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.054982    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-api-access-pn2cp" (OuterVolumeSpecName: "kube-api-access-pn2cp") pod "2a560e9d-2692-4715-a3cc-a6b195a40e2e" (UID: "2a560e9d-2692-4715-a3cc-a6b195a40e2e"). InnerVolumeSpecName "kube-api-access-pn2cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:16:16.153755 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.153670    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn2cp\" (UniqueName: \"kubernetes.io/projected/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-api-access-pn2cp\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:16.153755 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.153697    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a560e9d-2692-4715-a3cc-a6b195a40e2e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:16.153755 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.153708    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a560e9d-2692-4715-a3cc-a6b195a40e2e-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:16.384973 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.384935    2585 generic.go:356] "Generic (PLEG): container finished" podID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerID="e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05" exitCode=0
Jul 08 07:16:16.385138 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.384980    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" event={"ID":"2a560e9d-2692-4715-a3cc-a6b195a40e2e","Type":"ContainerDied","Data":"e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05"}
Jul 08 07:16:16.385138 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.384992    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"
Jul 08 07:16:16.385138 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.385041    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx" event={"ID":"2a560e9d-2692-4715-a3cc-a6b195a40e2e","Type":"ContainerDied","Data":"748e2cb4e11664582621b6aef43756e5fdfc86e4d98d19299abdd73a9d1ece1c"}
Jul 08 07:16:16.385138 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.385080    2585 scope.go:117] "RemoveContainer" containerID="34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7"
Jul 08 07:16:16.392975 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.392941    2585 scope.go:117] "RemoveContainer" containerID="e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05"
Jul 08 07:16:16.400074 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.400056    2585 scope.go:117] "RemoveContainer" containerID="34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7"
Jul 08 07:16:16.400289 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:16.400272    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7\": container with ID starting with 34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7 not found: ID does not exist" containerID="34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7"
Jul 08 07:16:16.400331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.400296    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7"} err="failed to get container status \"34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7\": rpc error: code = NotFound desc = could not find container \"34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7\": container with ID starting with 34b443b74fc8d4780ac6c2f0607e2b0b01aa9ef3394ccc088548507df497a9e7 not found: ID does not exist"
Jul 08 07:16:16.400331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.400310    2585 scope.go:117] "RemoveContainer" containerID="e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05"
Jul 08 07:16:16.400543 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:16.400519    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05\": container with ID starting with e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05 not found: ID does not exist" containerID="e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05"
Jul 08 07:16:16.400637 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.400549    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05"} err="failed to get container status \"e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05\": rpc error: code = NotFound desc = could not find container \"e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05\": container with ID starting with e342e015377d2119fe1ed5b3c1f46b20f1e94811139c6a9762da173720de6b05 not found: ID does not exist"
Jul 08 07:16:16.407182 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.407134    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"]
Jul 08 07:16:16.410425 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.410405    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-104b4-predictor-57dfd5fd5-9d5gx"]
Jul 08 07:16:16.949644 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:16.949613    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" path="/var/lib/kubelet/pods/2a560e9d-2692-4715-a3cc-a6b195a40e2e/volumes"
Jul 08 07:16:19.380218 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:19.380190    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:16:19.380710 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:19.380684    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 08 07:16:20.630367 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:20.630332    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:25.629706 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:25.629662    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:25.630077 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:25.629785    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:16:29.381065 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:29.381028    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 08 07:16:30.629815 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:30.629777    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:35.630261 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:35.630221    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:39.381100 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:39.381064    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 08 07:16:40.629840 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:40.629796    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:42.422704 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.422672    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"]
Jul 08 07:16:42.423075 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.422874    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" containerID="cri-o://5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214" gracePeriod=30
Jul 08 07:16:42.661961 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.661927    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"]
Jul 08 07:16:42.662381 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.662363    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kube-rbac-proxy"
Jul 08 07:16:42.662443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.662384    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kube-rbac-proxy"
Jul 08 07:16:42.662443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.662399    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container"
Jul 08 07:16:42.662443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.662408    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container"
Jul 08 07:16:42.662608 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.662488    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kube-rbac-proxy"
Jul 08 07:16:42.662608 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.662504    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a560e9d-2692-4715-a3cc-a6b195a40e2e" containerName="kserve-container"
Jul 08 07:16:42.665916 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.665894    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.669665 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.669642    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8ecaa-predictor-serving-cert\""
Jul 08 07:16:42.669768 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.669648    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8ecaa-kube-rbac-proxy-sar-config\""
Jul 08 07:16:42.676278 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.676255    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"]
Jul 08 07:16:42.695034 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.695013    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:16:42.717939 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.717916    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"]
Jul 08 07:16:42.718164 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.718138    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container" containerID="cri-o://8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97" gracePeriod=30
Jul 08 07:16:42.718443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.718300    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kube-rbac-proxy" containerID="cri-o://01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f" gracePeriod=30
Jul 08 07:16:42.759412 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.759385    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.759557 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.759428    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xs58\" (UniqueName: \"kubernetes.io/projected/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-api-access-6xs58\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.759557 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.759459    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.860083 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.860029    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls\") pod \"b115ebea-050e-472d-b394-8c5cf46ee772\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") "
Jul 08 07:16:42.860243 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.860103    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b115ebea-050e-472d-b394-8c5cf46ee772-openshift-service-ca-bundle\") pod \"b115ebea-050e-472d-b394-8c5cf46ee772\" (UID: \"b115ebea-050e-472d-b394-8c5cf46ee772\") "
Jul 08 07:16:42.860243 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.860211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.860351 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.860258    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.860351 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.860299    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xs58\" (UniqueName: \"kubernetes.io/projected/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-api-access-6xs58\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.860452 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:42.860431    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-serving-cert: secret "error-404-isvc-8ecaa-predictor-serving-cert" not found
Jul 08 07:16:42.860501 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.860457    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b115ebea-050e-472d-b394-8c5cf46ee772-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b115ebea-050e-472d-b394-8c5cf46ee772" (UID: "b115ebea-050e-472d-b394-8c5cf46ee772"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:16:42.860581 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:42.860517    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls podName:8c27b572-4317-42d9-86b1-1dd37781ce80 nodeName:}" failed. No retries permitted until 2026-07-08 07:16:43.360495508 +0000 UTC m=+1279.147331815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls") pod "error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" (UID: "8c27b572-4317-42d9-86b1-1dd37781ce80") : secret "error-404-isvc-8ecaa-predictor-serving-cert" not found
Jul 08 07:16:42.861043 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.861020    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.862393 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.862367    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b115ebea-050e-472d-b394-8c5cf46ee772" (UID: "b115ebea-050e-472d-b394-8c5cf46ee772"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:16:42.869285 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.869264    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xs58\" (UniqueName: \"kubernetes.io/projected/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-api-access-6xs58\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:42.961117 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.961089    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b115ebea-050e-472d-b394-8c5cf46ee772-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:42.961117 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:42.961116    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b115ebea-050e-472d-b394-8c5cf46ee772-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:43.365166 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.365063    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:43.367551 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.367534    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls\") pod \"error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") " pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:43.484982 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.484948    2585 generic.go:356] "Generic (PLEG): container finished" podID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerID="01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f" exitCode=2
Jul 08 07:16:43.485412 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.485012    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" event={"ID":"c67af24e-21d8-41f8-a84a-149dcc5350df","Type":"ContainerDied","Data":"01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f"}
Jul 08 07:16:43.486153 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.486131    2585 generic.go:356] "Generic (PLEG): container finished" podID="b115ebea-050e-472d-b394-8c5cf46ee772" containerID="0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059" exitCode=0
Jul 08 07:16:43.486259 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.486175    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"
Jul 08 07:16:43.486259 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.486196    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" event={"ID":"b115ebea-050e-472d-b394-8c5cf46ee772","Type":"ContainerDied","Data":"0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059"}
Jul 08 07:16:43.486259 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.486230    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6" event={"ID":"b115ebea-050e-472d-b394-8c5cf46ee772","Type":"ContainerDied","Data":"c70947a9ea73e51a44a87b98a7e00ddc85d94dd2a1c82059b007c442ccd37b48"}
Jul 08 07:16:43.486259 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.486251    2585 scope.go:117] "RemoveContainer" containerID="0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059"
Jul 08 07:16:43.493897 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.493872    2585 scope.go:117] "RemoveContainer" containerID="0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059"
Jul 08 07:16:43.494150 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:43.494132    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059\": container with ID starting with 0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059 not found: ID does not exist" containerID="0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059"
Jul 08 07:16:43.494222 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.494159    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059"} err="failed to get container status \"0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059\": rpc error: code = NotFound desc = could not find container \"0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059\": container with ID starting with 0342f89c1d9d3620ee5f34ff9ba1f208e0ad9df0ff151741b7375f8d90e0b059 not found: ID does not exist"
Jul 08 07:16:43.502929 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.502909    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"]
Jul 08 07:16:43.506187 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.506168    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-104b4-c874cb76f-zv7n6"]
Jul 08 07:16:43.591662 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.591615    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:43.725800 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:43.725775    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"]
Jul 08 07:16:43.727740 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:16:43.727709    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c27b572_4317_42d9_86b1_1dd37781ce80.slice/crio-1e229ac4953c33dd234fc58a493588240748a413cd16f940e590a7015518cbc3 WatchSource:0}: Error finding container 1e229ac4953c33dd234fc58a493588240748a413cd16f940e590a7015518cbc3: Status 404 returned error can't find the container with id 1e229ac4953c33dd234fc58a493588240748a413cd16f940e590a7015518cbc3
Jul 08 07:16:44.491350 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.491306    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" event={"ID":"8c27b572-4317-42d9-86b1-1dd37781ce80","Type":"ContainerStarted","Data":"620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d"}
Jul 08 07:16:44.491848 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.491412    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" event={"ID":"8c27b572-4317-42d9-86b1-1dd37781ce80","Type":"ContainerStarted","Data":"4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc"}
Jul 08 07:16:44.491848 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.491452    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:44.491848 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.491477    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:44.491848 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.491502    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" event={"ID":"8c27b572-4317-42d9-86b1-1dd37781ce80","Type":"ContainerStarted","Data":"1e229ac4953c33dd234fc58a493588240748a413cd16f940e590a7015518cbc3"}
Jul 08 07:16:44.492967 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.492937    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 08 07:16:44.511672 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.511621    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podStartSLOduration=2.511608114 podStartE2EDuration="2.511608114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:16:44.51102555 +0000 UTC m=+1280.297861891" watchObservedRunningTime="2026-07-08 07:16:44.511608114 +0000 UTC m=+1280.298444431"
Jul 08 07:16:44.949206 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:44.949177    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" path="/var/lib/kubelet/pods/b115ebea-050e-472d-b394-8c5cf46ee772/volumes"
Jul 08 07:16:45.497008 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.496967    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 08 07:16:45.737471 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.737434    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:45.868167 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.868143    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:16:45.983915 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.983881    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvqc7\" (UniqueName: \"kubernetes.io/projected/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-api-access-lvqc7\") pod \"c67af24e-21d8-41f8-a84a-149dcc5350df\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") "
Jul 08 07:16:45.983915 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.983919    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls\") pod \"c67af24e-21d8-41f8-a84a-149dcc5350df\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") "
Jul 08 07:16:45.984117 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.983961    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-rbac-proxy-sar-config\") pod \"c67af24e-21d8-41f8-a84a-149dcc5350df\" (UID: \"c67af24e-21d8-41f8-a84a-149dcc5350df\") "
Jul 08 07:16:45.984367 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.984344    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c67af24e-21d8-41f8-a84a-149dcc5350df" (UID: "c67af24e-21d8-41f8-a84a-149dcc5350df"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:16:45.985996 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.985962    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-api-access-lvqc7" (OuterVolumeSpecName: "kube-api-access-lvqc7") pod "c67af24e-21d8-41f8-a84a-149dcc5350df" (UID: "c67af24e-21d8-41f8-a84a-149dcc5350df"). InnerVolumeSpecName "kube-api-access-lvqc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:16:45.986114 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:45.986048    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c67af24e-21d8-41f8-a84a-149dcc5350df" (UID: "c67af24e-21d8-41f8-a84a-149dcc5350df"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:16:46.084882 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.084778    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvqc7\" (UniqueName: \"kubernetes.io/projected/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-api-access-lvqc7\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:46.084882 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.084825    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c67af24e-21d8-41f8-a84a-149dcc5350df-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:46.084882 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.084841    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c67af24e-21d8-41f8-a84a-149dcc5350df-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:16:46.501431 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.501395    2585 generic.go:356] "Generic (PLEG): container finished" podID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerID="8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97" exitCode=0
Jul 08 07:16:46.501824 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.501442    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"
Jul 08 07:16:46.501824 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.501485    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" event={"ID":"c67af24e-21d8-41f8-a84a-149dcc5350df","Type":"ContainerDied","Data":"8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97"}
Jul 08 07:16:46.501824 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.501542    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5" event={"ID":"c67af24e-21d8-41f8-a84a-149dcc5350df","Type":"ContainerDied","Data":"aaccb57cb059095be62726d0361a4549869e505c3f06e7e92857788c0e2cb895"}
Jul 08 07:16:46.501824 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.501586    2585 scope.go:117] "RemoveContainer" containerID="01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f"
Jul 08 07:16:46.509819 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.509802    2585 scope.go:117] "RemoveContainer" containerID="8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97"
Jul 08 07:16:46.516715 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.516699    2585 scope.go:117] "RemoveContainer" containerID="01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f"
Jul 08 07:16:46.516955 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:46.516937    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f\": container with ID starting with 01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f not found: ID does not exist" containerID="01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f"
Jul 08 07:16:46.517016 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.516964    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f"} err="failed to get container status \"01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f\": rpc error: code = NotFound desc = could not find container \"01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f\": container with ID starting with 01a0ae45093d75134dab7c6716dfa9788c829a61d38bf7db3b8ac002e02ea08f not found: ID does not exist"
Jul 08 07:16:46.517016 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.516980    2585 scope.go:117] "RemoveContainer" containerID="8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97"
Jul 08 07:16:46.517205 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:16:46.517188    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97\": container with ID starting with 8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97 not found: ID does not exist" containerID="8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97"
Jul 08 07:16:46.517251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.517213    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97"} err="failed to get container status \"8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97\": rpc error: code = NotFound desc = could not find container \"8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97\": container with ID starting with 8b0c31da91f77a303c0740da885d88499eb22604042faedd26277ee25f5aeb97 not found: ID does not exist"
Jul 08 07:16:46.522602 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.522582    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"]
Jul 08 07:16:46.529506 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.529480    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1469a-predictor-5dfdcd7c4f-tntr5"]
Jul 08 07:16:46.949368 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:46.949340    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" path="/var/lib/kubelet/pods/c67af24e-21d8-41f8-a84a-149dcc5350df/volumes"
Jul 08 07:16:49.381552 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:49.381518    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 08 07:16:50.501509 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:50.501478    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:16:50.501906 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:50.501881    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 08 07:16:50.736257 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:50.736211    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:55.735681 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:55.735640    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:16:55.736064 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:55.735780    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:16:59.381460 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:16:59.381430    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:17:00.502708 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:00.502670    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 08 07:17:00.736432 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:00.736389    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:05.736261 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:05.736218    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:10.502323 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:10.502282    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 08 07:17:10.735881 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:10.735840    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:12.563191 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.563166    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:17:12.571269 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.571250    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls\") pod \"7d48dba0-8ae4-4962-8405-0860ae24850e\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") "
Jul 08 07:17:12.571343 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.571333    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d48dba0-8ae4-4962-8405-0860ae24850e-openshift-service-ca-bundle\") pod \"7d48dba0-8ae4-4962-8405-0860ae24850e\" (UID: \"7d48dba0-8ae4-4962-8405-0860ae24850e\") "
Jul 08 07:17:12.571659 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.571634    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d48dba0-8ae4-4962-8405-0860ae24850e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "7d48dba0-8ae4-4962-8405-0860ae24850e" (UID: "7d48dba0-8ae4-4962-8405-0860ae24850e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:17:12.573258 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.573235    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7d48dba0-8ae4-4962-8405-0860ae24850e" (UID: "7d48dba0-8ae4-4962-8405-0860ae24850e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:17:12.596175 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.596144    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerID="5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214" exitCode=0
Jul 08 07:17:12.596314 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.596195    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"
Jul 08 07:17:12.596314 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.596226    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" event={"ID":"7d48dba0-8ae4-4962-8405-0860ae24850e","Type":"ContainerDied","Data":"5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214"}
Jul 08 07:17:12.596314 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.596307    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt" event={"ID":"7d48dba0-8ae4-4962-8405-0860ae24850e","Type":"ContainerDied","Data":"54583b00d1d1b5acbac4714c7bd952f1f44c28cd81100f5bf120a5f41bc30e76"}
Jul 08 07:17:12.596477 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.596336    2585 scope.go:117] "RemoveContainer" containerID="5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214"
Jul 08 07:17:12.604433 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.604419    2585 scope.go:117] "RemoveContainer" containerID="5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214"
Jul 08 07:17:12.604724 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:12.604707    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214\": container with ID starting with 5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214 not found: ID does not exist" containerID="5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214"
Jul 08 07:17:12.604818 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.604733    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214"} err="failed to get container status \"5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214\": rpc error: code = NotFound desc = could not find container \"5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214\": container with ID starting with 5cb5f5f64578afe9b59a82f257348733dffbd754bae549529533b10680f72214 not found: ID does not exist"
Jul 08 07:17:12.616373 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.616351    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"]
Jul 08 07:17:12.619746 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.619729    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1469a-d7c587ff8-7j2jt"]
Jul 08 07:17:12.672369 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.672336    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d48dba0-8ae4-4962-8405-0860ae24850e-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:12.672369 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.672364    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d48dba0-8ae4-4962-8405-0860ae24850e-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:12.823817 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.823780    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"]
Jul 08 07:17:12.824181 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824165    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824182    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824197    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824206    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824219    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824228    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824238    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kube-rbac-proxy"
Jul 08 07:17:12.824251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824246    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kube-rbac-proxy"
Jul 08 07:17:12.824482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824330    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b115ebea-050e-472d-b394-8c5cf46ee772" containerName="switch-graph-104b4"
Jul 08 07:17:12.824482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824343    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kserve-container"
Jul 08 07:17:12.824482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824352    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67af24e-21d8-41f8-a84a-149dcc5350df" containerName="kube-rbac-proxy"
Jul 08 07:17:12.824482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.824364    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" containerName="sequence-graph-1469a"
Jul 08 07:17:12.828602 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.828584    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:12.831326 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.831308    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-4baa0-serving-cert\""
Jul 08 07:17:12.831406 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.831375    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-4baa0-kube-rbac-proxy-sar-config\""
Jul 08 07:17:12.834955 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.834936    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"]
Jul 08 07:17:12.873527 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.873490    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f47063fa-4e8e-4c98-ad65-cc62f5858313-openshift-service-ca-bundle\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:12.873711 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.873538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:12.949954 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.949926    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d48dba0-8ae4-4962-8405-0860ae24850e" path="/var/lib/kubelet/pods/7d48dba0-8ae4-4962-8405-0860ae24850e/volumes"
Jul 08 07:17:12.974619 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.974587    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f47063fa-4e8e-4c98-ad65-cc62f5858313-openshift-service-ca-bundle\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:12.974775 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.974646    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:12.974775 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:12.974759    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-4baa0-serving-cert: secret "ensemble-graph-4baa0-serving-cert" not found
Jul 08 07:17:12.974885 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:12.974818    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls podName:f47063fa-4e8e-4c98-ad65-cc62f5858313 nodeName:}" failed. No retries permitted until 2026-07-08 07:17:13.474801862 +0000 UTC m=+1309.261638156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls") pod "ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" (UID: "f47063fa-4e8e-4c98-ad65-cc62f5858313") : secret "ensemble-graph-4baa0-serving-cert" not found
Jul 08 07:17:12.975443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:12.975422    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f47063fa-4e8e-4c98-ad65-cc62f5858313-openshift-service-ca-bundle\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:13.478222 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:13.478182    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:13.480672 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:13.480641    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls\") pod \"ensemble-graph-4baa0-5fb5d5fbd9-6jf7m\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") " pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:13.740095 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:13.740009    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:13.866262 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:13.866215    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"]
Jul 08 07:17:13.868814 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:17:13.868789    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47063fa_4e8e_4c98_ad65_cc62f5858313.slice/crio-ad74f1c7405123a817f17716ab1a1892b6fdfcd055f7cb2cc2eb0e59a1e3e2c4 WatchSource:0}: Error finding container ad74f1c7405123a817f17716ab1a1892b6fdfcd055f7cb2cc2eb0e59a1e3e2c4: Status 404 returned error can't find the container with id ad74f1c7405123a817f17716ab1a1892b6fdfcd055f7cb2cc2eb0e59a1e3e2c4
Jul 08 07:17:14.605924 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:14.605890    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" event={"ID":"f47063fa-4e8e-4c98-ad65-cc62f5858313","Type":"ContainerStarted","Data":"6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656"}
Jul 08 07:17:14.606182 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:14.605945    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" event={"ID":"f47063fa-4e8e-4c98-ad65-cc62f5858313","Type":"ContainerStarted","Data":"ad74f1c7405123a817f17716ab1a1892b6fdfcd055f7cb2cc2eb0e59a1e3e2c4"}
Jul 08 07:17:14.606182 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:14.605999    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:14.622853 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:14.622804    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podStartSLOduration=2.6227887 podStartE2EDuration="2.6227887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:17:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:17:14.621757877 +0000 UTC m=+1310.408594210" watchObservedRunningTime="2026-07-08 07:17:14.6227887 +0000 UTC m=+1310.409625016"
Jul 08 07:17:20.502177 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:20.502137    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 08 07:17:20.614640 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:20.614608    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:22.851532 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:22.851457    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"]
Jul 08 07:17:22.851966 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:22.851641    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" containerID="cri-o://6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656" gracePeriod=30
Jul 08 07:17:23.082540 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.082509    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"]
Jul 08 07:17:23.086084 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.086064    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.088916 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.088884    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1f938-predictor-serving-cert\""
Jul 08 07:17:23.089045 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.088918    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1f938-kube-rbac-proxy-sar-config\""
Jul 08 07:17:23.095509 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.095486    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"]
Jul 08 07:17:23.102206 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.102145    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"]
Jul 08 07:17:23.102398 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.102349    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container" containerID="cri-o://3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5" gracePeriod=30
Jul 08 07:17:23.102574 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.102382    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kube-rbac-proxy" containerID="cri-o://19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb" gracePeriod=30
Jul 08 07:17:23.153653 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.153619    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sdbz\" (UniqueName: \"kubernetes.io/projected/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-api-access-6sdbz\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.153811 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.153686    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.153811 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.153710    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.254985 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.254940    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sdbz\" (UniqueName: \"kubernetes.io/projected/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-api-access-6sdbz\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.255169 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.255022    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.255169 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.255045    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.255298 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:23.255162    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-1f938-predictor-serving-cert: secret "error-404-isvc-1f938-predictor-serving-cert" not found
Jul 08 07:17:23.255298 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:23.255250    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls podName:5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206 nodeName:}" failed. No retries permitted until 2026-07-08 07:17:23.755229001 +0000 UTC m=+1319.542065304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls") pod "error-404-isvc-1f938-predictor-59ff946b45-j22r5" (UID: "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206") : secret "error-404-isvc-1f938-predictor-serving-cert" not found
Jul 08 07:17:23.255799 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.255776    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.265470 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.265447    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sdbz\" (UniqueName: \"kubernetes.io/projected/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-api-access-6sdbz\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.645156 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.645120    2585 generic.go:356] "Generic (PLEG): container finished" podID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerID="19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb" exitCode=2
Jul 08 07:17:23.645300 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.645172    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" event={"ID":"5e698b4c-69c9-4188-8dac-709f47e602a4","Type":"ContainerDied","Data":"19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb"}
Jul 08 07:17:23.758283 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.758247    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.760604 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.760573    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls\") pod \"error-404-isvc-1f938-predictor-59ff946b45-j22r5\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") " pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:23.999248 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:23.999208    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:24.129375 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.129352    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"]
Jul 08 07:17:24.131199 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:17:24.131169    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa2e8aa_4b71_4cdf_8d37_3cd0ba5f9206.slice/crio-dc64037617b61ce2a531ea1a3ebc09d0fbb2a5a91a3fcb75b6b352512538e04d WatchSource:0}: Error finding container dc64037617b61ce2a531ea1a3ebc09d0fbb2a5a91a3fcb75b6b352512538e04d: Status 404 returned error can't find the container with id dc64037617b61ce2a531ea1a3ebc09d0fbb2a5a91a3fcb75b6b352512538e04d
Jul 08 07:17:24.377032 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.376932    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 08 07:17:24.650539 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.650453    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" event={"ID":"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206","Type":"ContainerStarted","Data":"087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2"}
Jul 08 07:17:24.650539 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.650514    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" event={"ID":"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206","Type":"ContainerStarted","Data":"0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be"}
Jul 08 07:17:24.650539 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.650535    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" event={"ID":"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206","Type":"ContainerStarted","Data":"dc64037617b61ce2a531ea1a3ebc09d0fbb2a5a91a3fcb75b6b352512538e04d"}
Jul 08 07:17:24.650827 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.650580    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:24.670143 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:24.670093    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podStartSLOduration=1.670077136 podStartE2EDuration="1.670077136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:17:24.668179601 +0000 UTC m=+1320.455015916" watchObservedRunningTime="2026-07-08 07:17:24.670077136 +0000 UTC m=+1320.456913451"
Jul 08 07:17:25.613377 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:25.613342    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:25.654023 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:25.653988    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:25.655201 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:25.655171    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 08 07:17:26.450482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.450456    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:17:26.478335 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.478306    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnz56\" (UniqueName: \"kubernetes.io/projected/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-api-access-dnz56\") pod \"5e698b4c-69c9-4188-8dac-709f47e602a4\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") "
Jul 08 07:17:26.478473 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.478381    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e698b4c-69c9-4188-8dac-709f47e602a4-proxy-tls\") pod \"5e698b4c-69c9-4188-8dac-709f47e602a4\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") "
Jul 08 07:17:26.478473 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.478439    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-rbac-proxy-sar-config\") pod \"5e698b4c-69c9-4188-8dac-709f47e602a4\" (UID: \"5e698b4c-69c9-4188-8dac-709f47e602a4\") "
Jul 08 07:17:26.478824 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.478791    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5e698b4c-69c9-4188-8dac-709f47e602a4" (UID: "5e698b4c-69c9-4188-8dac-709f47e602a4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:17:26.480431 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.480406    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e698b4c-69c9-4188-8dac-709f47e602a4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5e698b4c-69c9-4188-8dac-709f47e602a4" (UID: "5e698b4c-69c9-4188-8dac-709f47e602a4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:17:26.480530 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.480430    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-api-access-dnz56" (OuterVolumeSpecName: "kube-api-access-dnz56") pod "5e698b4c-69c9-4188-8dac-709f47e602a4" (UID: "5e698b4c-69c9-4188-8dac-709f47e602a4"). InnerVolumeSpecName "kube-api-access-dnz56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:17:26.579400 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.579357    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:26.579400 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.579391    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnz56\" (UniqueName: \"kubernetes.io/projected/5e698b4c-69c9-4188-8dac-709f47e602a4-kube-api-access-dnz56\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:26.579400 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.579401    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e698b4c-69c9-4188-8dac-709f47e602a4-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:26.658751 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.658720    2585 generic.go:356] "Generic (PLEG): container finished" podID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerID="3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5" exitCode=0
Jul 08 07:17:26.659132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.658768    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"
Jul 08 07:17:26.659132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.658809    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" event={"ID":"5e698b4c-69c9-4188-8dac-709f47e602a4","Type":"ContainerDied","Data":"3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5"}
Jul 08 07:17:26.659132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.658869    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj" event={"ID":"5e698b4c-69c9-4188-8dac-709f47e602a4","Type":"ContainerDied","Data":"6dbe3322b3e60cf12ada2ee2d013ce78193ff067747bd50cafa866089d104dd4"}
Jul 08 07:17:26.659132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.658899    2585 scope.go:117] "RemoveContainer" containerID="19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb"
Jul 08 07:17:26.659456 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.659424    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 08 07:17:26.666676 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.666661    2585 scope.go:117] "RemoveContainer" containerID="3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5"
Jul 08 07:17:26.673660 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.673642    2585 scope.go:117] "RemoveContainer" containerID="19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb"
Jul 08 07:17:26.673903 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:26.673885    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb\": container with ID starting with 19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb not found: ID does not exist" containerID="19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb"
Jul 08 07:17:26.673951 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.673912    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb"} err="failed to get container status \"19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb\": rpc error: code = NotFound desc = could not find container \"19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb\": container with ID starting with 19f50e896257fccc870bc66d61f78a541ae2c1d9def769c2f60bbd81d34522cb not found: ID does not exist"
Jul 08 07:17:26.673951 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.673929    2585 scope.go:117] "RemoveContainer" containerID="3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5"
Jul 08 07:17:26.674181 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:26.674166    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5\": container with ID starting with 3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5 not found: ID does not exist" containerID="3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5"
Jul 08 07:17:26.674232 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.674185    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5"} err="failed to get container status \"3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5\": rpc error: code = NotFound desc = could not find container \"3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5\": container with ID starting with 3fccff19099439333623d4942d0dab6ac9f4b573dcdbc0f0e198fac892aec3f5 not found: ID does not exist"
Jul 08 07:17:26.679770 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.679750    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"]
Jul 08 07:17:26.682503 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.682487    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4baa0-predictor-ff5686f95-v7hhj"]
Jul 08 07:17:26.949914 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:26.949882    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" path="/var/lib/kubelet/pods/5e698b4c-69c9-4188-8dac-709f47e602a4/volumes"
Jul 08 07:17:30.502681 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:30.502655    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:17:30.613015 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:30.612981    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:31.663901 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:31.663876    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:17:31.664348 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:31.664302    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 08 07:17:35.612866 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:35.612828    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:35.613233 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:35.612952    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:40.613026 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:40.612987    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:41.665139 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:41.665096    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 08 07:17:42.587221 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587187    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"]
Jul 08 07:17:42.587468 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587456    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container"
Jul 08 07:17:42.587514 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587469    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container"
Jul 08 07:17:42.587514 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587479    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kube-rbac-proxy"
Jul 08 07:17:42.587514 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587484    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kube-rbac-proxy"
Jul 08 07:17:42.587637 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587538    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kserve-container"
Jul 08 07:17:42.587637 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.587547    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e698b4c-69c9-4188-8dac-709f47e602a4" containerName="kube-rbac-proxy"
Jul 08 07:17:42.591761 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.591745    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:42.596932 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.596906    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8ecaa-kube-rbac-proxy-sar-config\""
Jul 08 07:17:42.597077 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.597058    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8ecaa-serving-cert\""
Jul 08 07:17:42.600083 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.600060    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"]
Jul 08 07:17:42.714868 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.714837    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:42.715235 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.714907    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf72db1e-4307-4102-8491-38813f98abff-openshift-service-ca-bundle\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:42.815702 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.815663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:42.815867 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.815831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf72db1e-4307-4102-8491-38813f98abff-openshift-service-ca-bundle\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:42.815909 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:42.815862    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-8ecaa-serving-cert: secret "sequence-graph-8ecaa-serving-cert" not found
Jul 08 07:17:42.815953 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:42.815941    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls podName:cf72db1e-4307-4102-8491-38813f98abff nodeName:}" failed. No retries permitted until 2026-07-08 07:17:43.315916517 +0000 UTC m=+1339.102752819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls") pod "sequence-graph-8ecaa-744b549df5-m5nct" (UID: "cf72db1e-4307-4102-8491-38813f98abff") : secret "sequence-graph-8ecaa-serving-cert" not found
Jul 08 07:17:42.816437 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:42.816421    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf72db1e-4307-4102-8491-38813f98abff-openshift-service-ca-bundle\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:43.320381 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.320331    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:43.322758 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.322734    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls\") pod \"sequence-graph-8ecaa-744b549df5-m5nct\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") " pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:43.502619 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.502556    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:43.625845 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.625816    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"]
Jul 08 07:17:43.628945 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:17:43.628915    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf72db1e_4307_4102_8491_38813f98abff.slice/crio-1a0af5567645468ced7d4dc97d9dcc94010c0a29d19ab78f9153c7e6e13b6bf5 WatchSource:0}: Error finding container 1a0af5567645468ced7d4dc97d9dcc94010c0a29d19ab78f9153c7e6e13b6bf5: Status 404 returned error can't find the container with id 1a0af5567645468ced7d4dc97d9dcc94010c0a29d19ab78f9153c7e6e13b6bf5
Jul 08 07:17:43.717410 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.717376    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" event={"ID":"cf72db1e-4307-4102-8491-38813f98abff","Type":"ContainerStarted","Data":"d2257d17b67fc399ec499c397e3bf7e078cadb48091f16306ff538404a1b38af"}
Jul 08 07:17:43.717799 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.717454    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" event={"ID":"cf72db1e-4307-4102-8491-38813f98abff","Type":"ContainerStarted","Data":"1a0af5567645468ced7d4dc97d9dcc94010c0a29d19ab78f9153c7e6e13b6bf5"}
Jul 08 07:17:43.717799 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.717501    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:43.734905 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:43.734852    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podStartSLOduration=1.7348366579999999 podStartE2EDuration="1.734836658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:17:43.733061379 +0000 UTC m=+1339.519897714" watchObservedRunningTime="2026-07-08 07:17:43.734836658 +0000 UTC m=+1339.521672975"
Jul 08 07:17:45.613275 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:45.613238    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:49.727970 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:49.727893    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:17:50.613017 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:50.612970    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:51.664630 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:51.664587    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 08 07:17:52.688404 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.688374    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"]
Jul 08 07:17:52.688851 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.688587    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" containerID="cri-o://d2257d17b67fc399ec499c397e3bf7e078cadb48091f16306ff538404a1b38af" gracePeriod=30
Jul 08 07:17:52.903767 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.903735    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"]
Jul 08 07:17:52.903990 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.903964    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container" containerID="cri-o://4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc" gracePeriod=30
Jul 08 07:17:52.904064 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.903992    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kube-rbac-proxy" containerID="cri-o://620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d" gracePeriod=30
Jul 08 07:17:52.922312 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.922283    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"]
Jul 08 07:17:52.925715 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.925557    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:52.928472 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.928433    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3d727-kube-rbac-proxy-sar-config\""
Jul 08 07:17:52.929282 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.928685    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3d727-predictor-serving-cert\""
Jul 08 07:17:52.935529 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.934698    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"]
Jul 08 07:17:52.997285 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.997259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:52.997419 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.997298    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b26xn\" (UniqueName: \"kubernetes.io/projected/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-api-access-b26xn\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:52.997419 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:52.997363    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.024999 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.024979    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:53.098679 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.098644    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls\") pod \"f47063fa-4e8e-4c98-ad65-cc62f5858313\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") "
Jul 08 07:17:53.098862 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.098698    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f47063fa-4e8e-4c98-ad65-cc62f5858313-openshift-service-ca-bundle\") pod \"f47063fa-4e8e-4c98-ad65-cc62f5858313\" (UID: \"f47063fa-4e8e-4c98-ad65-cc62f5858313\") "
Jul 08 07:17:53.098862 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.098768    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.098862 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.098811    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b26xn\" (UniqueName: \"kubernetes.io/projected/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-api-access-b26xn\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.099024 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.098897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.099024 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:53.098910    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-3d727-predictor-serving-cert: secret "error-404-isvc-3d727-predictor-serving-cert" not found
Jul 08 07:17:53.099024 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:53.098998    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls podName:1eba7aca-26ed-432d-b809-9a0aaed29bf0 nodeName:}" failed. No retries permitted until 2026-07-08 07:17:53.598975983 +0000 UTC m=+1349.385812294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls") pod "error-404-isvc-3d727-predictor-58657f85cc-bp9xf" (UID: "1eba7aca-26ed-432d-b809-9a0aaed29bf0") : secret "error-404-isvc-3d727-predictor-serving-cert" not found
Jul 08 07:17:53.099180 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.099097    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47063fa-4e8e-4c98-ad65-cc62f5858313-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "f47063fa-4e8e-4c98-ad65-cc62f5858313" (UID: "f47063fa-4e8e-4c98-ad65-cc62f5858313"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:17:53.099865 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.099845    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.101267 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.101244    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f47063fa-4e8e-4c98-ad65-cc62f5858313" (UID: "f47063fa-4e8e-4c98-ad65-cc62f5858313"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:17:53.107743 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.107722    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b26xn\" (UniqueName: \"kubernetes.io/projected/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-api-access-b26xn\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.200016 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.199975    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f47063fa-4e8e-4c98-ad65-cc62f5858313-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:53.200016 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.200010    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f47063fa-4e8e-4c98-ad65-cc62f5858313-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:53.603896 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.603859    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.606155 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.606134    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls\") pod \"error-404-isvc-3d727-predictor-58657f85cc-bp9xf\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") " pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.754116 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.754083    2585 generic.go:356] "Generic (PLEG): container finished" podID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerID="620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d" exitCode=2
Jul 08 07:17:53.754535 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.754154    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" event={"ID":"8c27b572-4317-42d9-86b1-1dd37781ce80","Type":"ContainerDied","Data":"620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d"}
Jul 08 07:17:53.755250 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.755230    2585 generic.go:356] "Generic (PLEG): container finished" podID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerID="6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656" exitCode=0
Jul 08 07:17:53.755331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.755281    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"
Jul 08 07:17:53.755331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.755283    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" event={"ID":"f47063fa-4e8e-4c98-ad65-cc62f5858313","Type":"ContainerDied","Data":"6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656"}
Jul 08 07:17:53.755331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.755328    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m" event={"ID":"f47063fa-4e8e-4c98-ad65-cc62f5858313","Type":"ContainerDied","Data":"ad74f1c7405123a817f17716ab1a1892b6fdfcd055f7cb2cc2eb0e59a1e3e2c4"}
Jul 08 07:17:53.755435 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.755350    2585 scope.go:117] "RemoveContainer" containerID="6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656"
Jul 08 07:17:53.763125 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.763106    2585 scope.go:117] "RemoveContainer" containerID="6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656"
Jul 08 07:17:53.763414 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:53.763396    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656\": container with ID starting with 6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656 not found: ID does not exist" containerID="6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656"
Jul 08 07:17:53.763496 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.763422    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656"} err="failed to get container status \"6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656\": rpc error: code = NotFound desc = could not find container \"6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656\": container with ID starting with 6ef355eacfae18db18d41d9b46b744d1be74e2e795b92047f69856be1b0bf656 not found: ID does not exist"
Jul 08 07:17:53.775693 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.775664    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"]
Jul 08 07:17:53.779422 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.779399    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-4baa0-5fb5d5fbd9-6jf7m"]
Jul 08 07:17:53.839433 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.839400    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:53.969514 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:53.969353    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"]
Jul 08 07:17:53.971986 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:17:53.971957    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eba7aca_26ed_432d_b809_9a0aaed29bf0.slice/crio-27310753170fca9990d78ab4a2cddef6052288a033ab8fc1fd8231a574cd7615 WatchSource:0}: Error finding container 27310753170fca9990d78ab4a2cddef6052288a033ab8fc1fd8231a574cd7615: Status 404 returned error can't find the container with id 27310753170fca9990d78ab4a2cddef6052288a033ab8fc1fd8231a574cd7615
Jul 08 07:17:54.726324 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.726291    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:17:54.768091 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.768058    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" event={"ID":"1eba7aca-26ed-432d-b809-9a0aaed29bf0","Type":"ContainerStarted","Data":"e5d0767c6143ff1ea207c20a97d5aeba7aa5ab840de1f6d708b08bb0ef41047c"}
Jul 08 07:17:54.768513 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.768136    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" event={"ID":"1eba7aca-26ed-432d-b809-9a0aaed29bf0","Type":"ContainerStarted","Data":"849f58e2ac4655c544963b30969316155fa387ef9c60d9366399c880475162f1"}
Jul 08 07:17:54.768513 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.768156    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" event={"ID":"1eba7aca-26ed-432d-b809-9a0aaed29bf0","Type":"ContainerStarted","Data":"27310753170fca9990d78ab4a2cddef6052288a033ab8fc1fd8231a574cd7615"}
Jul 08 07:17:54.768513 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.768322    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:54.768513 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.768376    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:17:54.769448 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.769426    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 08 07:17:54.788533 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.788485    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podStartSLOduration=2.788472679 podStartE2EDuration="2.788472679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:17:54.787403447 +0000 UTC m=+1350.574239763" watchObservedRunningTime="2026-07-08 07:17:54.788472679 +0000 UTC m=+1350.575308994"
Jul 08 07:17:54.949903 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:54.949863    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" path="/var/lib/kubelet/pods/f47063fa-4e8e-4c98-ad65-cc62f5858313/volumes"
Jul 08 07:17:55.498194 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:55.498152    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 08 07:17:55.771442 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:55.771338    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 08 07:17:56.245272 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.245249    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:17:56.324674 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.324581    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls\") pod \"8c27b572-4317-42d9-86b1-1dd37781ce80\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") "
Jul 08 07:17:56.324674 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.324652    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xs58\" (UniqueName: \"kubernetes.io/projected/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-api-access-6xs58\") pod \"8c27b572-4317-42d9-86b1-1dd37781ce80\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") "
Jul 08 07:17:56.324907 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.324702    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-rbac-proxy-sar-config\") pod \"8c27b572-4317-42d9-86b1-1dd37781ce80\" (UID: \"8c27b572-4317-42d9-86b1-1dd37781ce80\") "
Jul 08 07:17:56.325121 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.325090    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8c27b572-4317-42d9-86b1-1dd37781ce80" (UID: "8c27b572-4317-42d9-86b1-1dd37781ce80"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:17:56.326632 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.326610    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-api-access-6xs58" (OuterVolumeSpecName: "kube-api-access-6xs58") pod "8c27b572-4317-42d9-86b1-1dd37781ce80" (UID: "8c27b572-4317-42d9-86b1-1dd37781ce80"). InnerVolumeSpecName "kube-api-access-6xs58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:17:56.326700 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.326645    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8c27b572-4317-42d9-86b1-1dd37781ce80" (UID: "8c27b572-4317-42d9-86b1-1dd37781ce80"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:17:56.425935 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.425896    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xs58\" (UniqueName: \"kubernetes.io/projected/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-api-access-6xs58\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:56.425935 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.425928    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c27b572-4317-42d9-86b1-1dd37781ce80-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:56.425935 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.425938    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c27b572-4317-42d9-86b1-1dd37781ce80-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:17:56.776947 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.776910    2585 generic.go:356] "Generic (PLEG): container finished" podID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerID="4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc" exitCode=0
Jul 08 07:17:56.777331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.776962    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"
Jul 08 07:17:56.777331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.777000    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" event={"ID":"8c27b572-4317-42d9-86b1-1dd37781ce80","Type":"ContainerDied","Data":"4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc"}
Jul 08 07:17:56.777331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.777052    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd" event={"ID":"8c27b572-4317-42d9-86b1-1dd37781ce80","Type":"ContainerDied","Data":"1e229ac4953c33dd234fc58a493588240748a413cd16f940e590a7015518cbc3"}
Jul 08 07:17:56.777331 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.777077    2585 scope.go:117] "RemoveContainer" containerID="620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d"
Jul 08 07:17:56.785746 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.785727    2585 scope.go:117] "RemoveContainer" containerID="4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc"
Jul 08 07:17:56.792804 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.792780    2585 scope.go:117] "RemoveContainer" containerID="620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d"
Jul 08 07:17:56.793061 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:56.793042    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d\": container with ID starting with 620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d not found: ID does not exist" containerID="620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d"
Jul 08 07:17:56.793111 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.793070    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d"} err="failed to get container status \"620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d\": rpc error: code = NotFound desc = could not find container \"620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d\": container with ID starting with 620a6f70428afeff97bffc2d7c37d253b616d2d71e4889cbc437c9b926436e2d not found: ID does not exist"
Jul 08 07:17:56.793111 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.793087    2585 scope.go:117] "RemoveContainer" containerID="4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc"
Jul 08 07:17:56.793308 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:17:56.793291    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc\": container with ID starting with 4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc not found: ID does not exist" containerID="4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc"
Jul 08 07:17:56.793348 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.793314    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc"} err="failed to get container status \"4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc\": rpc error: code = NotFound desc = could not find container \"4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc\": container with ID starting with 4b11511beac966a17753e47c725bab4bd5963e1335b54617ced921bf7c8b06fc not found: ID does not exist"
Jul 08 07:17:56.799336 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.799316    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"]
Jul 08 07:17:56.801759 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.801729    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8ecaa-predictor-79f5d859c9-j8vtd"]
Jul 08 07:17:56.949087 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:56.949059    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" path="/var/lib/kubelet/pods/8c27b572-4317-42d9-86b1-1dd37781ce80/volumes"
Jul 08 07:17:59.725374 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:17:59.725339    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:18:00.776917 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:00.776888    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:18:00.777450 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:00.777417    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 08 07:18:01.664446 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:01.664410    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 08 07:18:04.725903 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:04.725867    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:18:04.726290 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:04.725980    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:18:09.725866 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:09.725827    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:18:10.777591 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:10.777538    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 08 07:18:11.664974 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:11.664941    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:18:14.726143 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:14.726108    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:18:19.725603 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:19.725551    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:18:20.777531 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:20.777490    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 08 07:18:22.871004 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.870971    2585 generic.go:356] "Generic (PLEG): container finished" podID="cf72db1e-4307-4102-8491-38813f98abff" containerID="d2257d17b67fc399ec499c397e3bf7e078cadb48091f16306ff538404a1b38af" exitCode=0
Jul 08 07:18:22.871304 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.871035    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" event={"ID":"cf72db1e-4307-4102-8491-38813f98abff","Type":"ContainerDied","Data":"d2257d17b67fc399ec499c397e3bf7e078cadb48091f16306ff538404a1b38af"}
Jul 08 07:18:22.871304 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.871077    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct" event={"ID":"cf72db1e-4307-4102-8491-38813f98abff","Type":"ContainerDied","Data":"1a0af5567645468ced7d4dc97d9dcc94010c0a29d19ab78f9153c7e6e13b6bf5"}
Jul 08 07:18:22.871304 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.871096    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a0af5567645468ced7d4dc97d9dcc94010c0a29d19ab78f9153c7e6e13b6bf5"
Jul 08 07:18:22.872148 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.872133    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:18:22.925726 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.925699    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls\") pod \"cf72db1e-4307-4102-8491-38813f98abff\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") "
Jul 08 07:18:22.925872 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.925759    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf72db1e-4307-4102-8491-38813f98abff-openshift-service-ca-bundle\") pod \"cf72db1e-4307-4102-8491-38813f98abff\" (UID: \"cf72db1e-4307-4102-8491-38813f98abff\") "
Jul 08 07:18:22.926121 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.926095    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf72db1e-4307-4102-8491-38813f98abff-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cf72db1e-4307-4102-8491-38813f98abff" (UID: "cf72db1e-4307-4102-8491-38813f98abff"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:18:22.927519 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:22.927501    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cf72db1e-4307-4102-8491-38813f98abff" (UID: "cf72db1e-4307-4102-8491-38813f98abff"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:18:23.026194 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.026161    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf72db1e-4307-4102-8491-38813f98abff-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:18:23.026194 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.026188    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf72db1e-4307-4102-8491-38813f98abff-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:18:23.078904 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.078822    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"]
Jul 08 07:18:23.079112 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079098    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079114    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079127    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kube-rbac-proxy"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079132    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kube-rbac-proxy"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079143    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079150    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079156    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa"
Jul 08 07:18:23.079179 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079161    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa"
Jul 08 07:18:23.079516 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079209    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kserve-container"
Jul 08 07:18:23.079516 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079218    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47063fa-4e8e-4c98-ad65-cc62f5858313" containerName="ensemble-graph-4baa0"
Jul 08 07:18:23.079516 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079227    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf72db1e-4307-4102-8491-38813f98abff" containerName="sequence-graph-8ecaa"
Jul 08 07:18:23.079516 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.079233    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c27b572-4317-42d9-86b1-1dd37781ce80" containerName="kube-rbac-proxy"
Jul 08 07:18:23.082057 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.082041    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.084700 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.084680    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1f938-kube-rbac-proxy-sar-config\""
Jul 08 07:18:23.084855 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.084838    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1f938-serving-cert\""
Jul 08 07:18:23.089573 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.089537    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"]
Jul 08 07:18:23.126616 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.126580    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.126784 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.126660    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d59aed-5bab-455e-8630-7d19e7af4636-openshift-service-ca-bundle\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.227189 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.227157    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.227395 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.227208    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d59aed-5bab-455e-8630-7d19e7af4636-openshift-service-ca-bundle\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.227395 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:18:23.227306    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-1f938-serving-cert: secret "ensemble-graph-1f938-serving-cert" not found
Jul 08 07:18:23.227395 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:18:23.227367    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls podName:c0d59aed-5bab-455e-8630-7d19e7af4636 nodeName:}" failed. No retries permitted until 2026-07-08 07:18:23.727350677 +0000 UTC m=+1379.514186970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls") pod "ensemble-graph-1f938-77484598f8-rppn4" (UID: "c0d59aed-5bab-455e-8630-7d19e7af4636") : secret "ensemble-graph-1f938-serving-cert" not found
Jul 08 07:18:23.227878 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.227859    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d59aed-5bab-455e-8630-7d19e7af4636-openshift-service-ca-bundle\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.731430 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.731389    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.733725 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.733704    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls\") pod \"ensemble-graph-1f938-77484598f8-rppn4\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") " pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:23.875020 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.874987    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"
Jul 08 07:18:23.891016 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.890988    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"]
Jul 08 07:18:23.896817 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.896793    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8ecaa-744b549df5-m5nct"]
Jul 08 07:18:23.992132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:23.992054    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:24.117230 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:24.117209    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"]
Jul 08 07:18:24.119864 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:18:24.119827    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d59aed_5bab_455e_8630_7d19e7af4636.slice/crio-93c60a171e1210d63d07b9c95bc64aef6dd0f9e6c489808f4dda9aafc9934ce0 WatchSource:0}: Error finding container 93c60a171e1210d63d07b9c95bc64aef6dd0f9e6c489808f4dda9aafc9934ce0: Status 404 returned error can't find the container with id 93c60a171e1210d63d07b9c95bc64aef6dd0f9e6c489808f4dda9aafc9934ce0
Jul 08 07:18:24.879132 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:24.879100    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" event={"ID":"c0d59aed-5bab-455e-8630-7d19e7af4636","Type":"ContainerStarted","Data":"17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6"}
Jul 08 07:18:24.879474 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:24.879151    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" event={"ID":"c0d59aed-5bab-455e-8630-7d19e7af4636","Type":"ContainerStarted","Data":"93c60a171e1210d63d07b9c95bc64aef6dd0f9e6c489808f4dda9aafc9934ce0"}
Jul 08 07:18:24.879474 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:24.879184    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:24.895834 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:24.895795    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podStartSLOduration=1.895781317 podStartE2EDuration="1.895781317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:18:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:18:24.895159002 +0000 UTC m=+1380.681995320" watchObservedRunningTime="2026-07-08 07:18:24.895781317 +0000 UTC m=+1380.682617632"
Jul 08 07:18:24.948903 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:24.948879    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf72db1e-4307-4102-8491-38813f98abff" path="/var/lib/kubelet/pods/cf72db1e-4307-4102-8491-38813f98abff/volumes"
Jul 08 07:18:30.777843 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:30.777801    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 08 07:18:30.888817 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:30.888780    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:18:40.778534 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:40.778507    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:18:52.899931 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.899896    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"]
Jul 08 07:18:52.904376 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.904359    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:52.907076 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.907059    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-3d727-kube-rbac-proxy-sar-config\""
Jul 08 07:18:52.907144 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.907104    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-3d727-serving-cert\""
Jul 08 07:18:52.913233 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.913209    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"]
Jul 08 07:18:52.946811 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.946777    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fd6661e-00cc-44aa-9516-9f2055ea572c-openshift-service-ca-bundle\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:52.946811 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:52.946817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.047668 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.047627    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fd6661e-00cc-44aa-9516-9f2055ea572c-openshift-service-ca-bundle\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.047668 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.047671    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.047922 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:18:53.047760    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-3d727-serving-cert: secret "sequence-graph-3d727-serving-cert" not found
Jul 08 07:18:53.047922 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:18:53.047829    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls podName:2fd6661e-00cc-44aa-9516-9f2055ea572c nodeName:}" failed. No retries permitted until 2026-07-08 07:18:53.547810472 +0000 UTC m=+1409.334646767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls") pod "sequence-graph-3d727-6dd4fd7669-crrj5" (UID: "2fd6661e-00cc-44aa-9516-9f2055ea572c") : secret "sequence-graph-3d727-serving-cert" not found
Jul 08 07:18:53.048281 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.048262    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fd6661e-00cc-44aa-9516-9f2055ea572c-openshift-service-ca-bundle\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.551266 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.551226    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.553656 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.553639    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls\") pod \"sequence-graph-3d727-6dd4fd7669-crrj5\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") " pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.815251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.815155    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:53.940901 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.940871    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"]
Jul 08 07:18:53.943403 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:18:53.943375    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd6661e_00cc_44aa_9516_9f2055ea572c.slice/crio-f2346c181cc3d1cb6bbdb77cd4cc4dff09310c63e8bdaef3151713281e512ecc WatchSource:0}: Error finding container f2346c181cc3d1cb6bbdb77cd4cc4dff09310c63e8bdaef3151713281e512ecc: Status 404 returned error can't find the container with id f2346c181cc3d1cb6bbdb77cd4cc4dff09310c63e8bdaef3151713281e512ecc
Jul 08 07:18:53.977864 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:53.977833    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" event={"ID":"2fd6661e-00cc-44aa-9516-9f2055ea572c","Type":"ContainerStarted","Data":"f2346c181cc3d1cb6bbdb77cd4cc4dff09310c63e8bdaef3151713281e512ecc"}
Jul 08 07:18:54.982076 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:54.982043    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" event={"ID":"2fd6661e-00cc-44aa-9516-9f2055ea572c","Type":"ContainerStarted","Data":"024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5"}
Jul 08 07:18:54.982463 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:54.982101    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:18:54.998518 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:18:54.998471    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podStartSLOduration=2.998455944 podStartE2EDuration="2.998455944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:18:54.997635445 +0000 UTC m=+1410.784471774" watchObservedRunningTime="2026-07-08 07:18:54.998455944 +0000 UTC m=+1410.785292260"
Jul 08 07:19:00.991279 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:19:00.991247    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:24:24.866926 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:24:24.866893    2585 scope.go:117] "RemoveContainer" containerID="d2257d17b67fc399ec499c397e3bf7e078cadb48091f16306ff538404a1b38af"
Jul 08 07:26:37.859745 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:37.859715    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"]
Jul 08 07:26:37.860229 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:37.859920    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" containerID="cri-o://17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6" gracePeriod=30
Jul 08 07:26:38.165233 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.165153    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"]
Jul 08 07:26:38.168816 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.168799    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.171670 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.171648    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-71ab7-kube-rbac-proxy-sar-config\""
Jul 08 07:26:38.171840 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.171828    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-71ab7-predictor-serving-cert\""
Jul 08 07:26:38.177393 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.177371    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"]
Jul 08 07:26:38.210170 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.210134    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"]
Jul 08 07:26:38.210418 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.210390    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container" containerID="cri-o://0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be" gracePeriod=30
Jul 08 07:26:38.210512 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.210410    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kube-rbac-proxy" containerID="cri-o://087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2" gracePeriod=30
Jul 08 07:26:38.276334 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.276298    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.276482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.276358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.276482 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.276393    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm9sc\" (UniqueName: \"kubernetes.io/projected/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-api-access-jm9sc\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.377475 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.377443    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.377475 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.377478    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm9sc\" (UniqueName: \"kubernetes.io/projected/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-api-access-jm9sc\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.377727 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.377530    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.377727 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:26:38.377674    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-serving-cert: secret "error-404-isvc-71ab7-predictor-serving-cert" not found
Jul 08 07:26:38.377822 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:26:38.377739    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls podName:73d21b91-ffca-4f69-9520-f46c9f9f2259 nodeName:}" failed. No retries permitted until 2026-07-08 07:26:38.877719688 +0000 UTC m=+1874.664556001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls") pod "error-404-isvc-71ab7-predictor-868947f459-fxsgw" (UID: "73d21b91-ffca-4f69-9520-f46c9f9f2259") : secret "error-404-isvc-71ab7-predictor-serving-cert" not found
Jul 08 07:26:38.378113 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.378090    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.388890 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.388864    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm9sc\" (UniqueName: \"kubernetes.io/projected/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-api-access-jm9sc\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.612376 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.612345    2585 generic.go:356] "Generic (PLEG): container finished" podID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerID="087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2" exitCode=2
Jul 08 07:26:38.612376 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.612381    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" event={"ID":"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206","Type":"ContainerDied","Data":"087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2"}
Jul 08 07:26:38.882630 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.882537    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:38.884885 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:38.884857    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls\") pod \"error-404-isvc-71ab7-predictor-868947f459-fxsgw\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") " pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:39.080260 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.080225    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:39.210683 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.210594    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"]
Jul 08 07:26:39.213255 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:26:39.213230    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d21b91_ffca_4f69_9520_f46c9f9f2259.slice/crio-aeb9ce507d8127a82267dd500d3fdc9ce32a760a26da452dd09e1b2bce3ac0f4 WatchSource:0}: Error finding container aeb9ce507d8127a82267dd500d3fdc9ce32a760a26da452dd09e1b2bce3ac0f4: Status 404 returned error can't find the container with id aeb9ce507d8127a82267dd500d3fdc9ce32a760a26da452dd09e1b2bce3ac0f4
Jul 08 07:26:39.618072 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.618034    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" event={"ID":"73d21b91-ffca-4f69-9520-f46c9f9f2259","Type":"ContainerStarted","Data":"c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1"}
Jul 08 07:26:39.618244 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.618119    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" event={"ID":"73d21b91-ffca-4f69-9520-f46c9f9f2259","Type":"ContainerStarted","Data":"7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7"}
Jul 08 07:26:39.618244 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.618150    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" event={"ID":"73d21b91-ffca-4f69-9520-f46c9f9f2259","Type":"ContainerStarted","Data":"aeb9ce507d8127a82267dd500d3fdc9ce32a760a26da452dd09e1b2bce3ac0f4"}
Jul 08 07:26:39.618244 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.618181    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:39.637349 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:39.637302    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podStartSLOduration=1.6372821960000001 podStartE2EDuration="1.637282196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:26:39.636030178 +0000 UTC m=+1875.422866513" watchObservedRunningTime="2026-07-08 07:26:39.637282196 +0000 UTC m=+1875.424118506"
Jul 08 07:26:40.621701 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:40.621660    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:40.622936 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:40.622913    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 08 07:26:40.887086 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:40.886988    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:26:41.464099 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.464077    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:26:41.606425 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.606341    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-rbac-proxy-sar-config\") pod \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") "
Jul 08 07:26:41.606425 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.606419    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls\") pod \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") "
Jul 08 07:26:41.606647 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.606516    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sdbz\" (UniqueName: \"kubernetes.io/projected/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-api-access-6sdbz\") pod \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\" (UID: \"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206\") "
Jul 08 07:26:41.606746 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.606726    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" (UID: "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:26:41.608556 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.608514    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-api-access-6sdbz" (OuterVolumeSpecName: "kube-api-access-6sdbz") pod "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" (UID: "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206"). InnerVolumeSpecName "kube-api-access-6sdbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:26:41.608556 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.608551    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" (UID: "5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:26:41.626104 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.626073    2585 generic.go:356] "Generic (PLEG): container finished" podID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerID="0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be" exitCode=0
Jul 08 07:26:41.626440 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.626142    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"
Jul 08 07:26:41.626440 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.626159    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" event={"ID":"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206","Type":"ContainerDied","Data":"0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be"}
Jul 08 07:26:41.626440 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.626214    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5" event={"ID":"5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206","Type":"ContainerDied","Data":"dc64037617b61ce2a531ea1a3ebc09d0fbb2a5a91a3fcb75b6b352512538e04d"}
Jul 08 07:26:41.626440 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.626239    2585 scope.go:117] "RemoveContainer" containerID="087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2"
Jul 08 07:26:41.626915 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.626891    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 08 07:26:41.637973 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.637953    2585 scope.go:117] "RemoveContainer" containerID="0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be"
Jul 08 07:26:41.645283 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.645266    2585 scope.go:117] "RemoveContainer" containerID="087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2"
Jul 08 07:26:41.645578 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:26:41.645539    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2\": container with ID starting with 087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2 not found: ID does not exist" containerID="087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2"
Jul 08 07:26:41.645667 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.645585    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2"} err="failed to get container status \"087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2\": rpc error: code = NotFound desc = could not find container \"087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2\": container with ID starting with 087a29be2763bd85625f3acb83fc52eec9b6eee2ab6845d9d331cb82123f0cf2 not found: ID does not exist"
Jul 08 07:26:41.645667 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.645601    2585 scope.go:117] "RemoveContainer" containerID="0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be"
Jul 08 07:26:41.645822 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:26:41.645806    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be\": container with ID starting with 0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be not found: ID does not exist" containerID="0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be"
Jul 08 07:26:41.645862 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.645826    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be"} err="failed to get container status \"0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be\": rpc error: code = NotFound desc = could not find container \"0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be\": container with ID starting with 0061a37a83cbce048c55a397e6701a277d7e0cb3106e9c14ac71bd20a88b33be not found: ID does not exist"
Jul 08 07:26:41.652858 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.652837    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"]
Jul 08 07:26:41.654676 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.654655    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1f938-predictor-59ff946b45-j22r5"]
Jul 08 07:26:41.707512 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.707482    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:26:41.707512 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.707509    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sdbz\" (UniqueName: \"kubernetes.io/projected/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-api-access-6sdbz\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:26:41.707755 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:41.707523    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:26:42.949338 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:42.949302    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" path="/var/lib/kubelet/pods/5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206/volumes"
Jul 08 07:26:45.887213 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:45.887172    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:26:46.632018 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:46.631988    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:26:46.632700 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:46.632669    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 08 07:26:50.886703 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:50.886664    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:26:50.887114 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:50.886777    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:26:55.887208 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:55.887161    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:26:56.633167 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:26:56.633128    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 08 07:27:00.886825 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:00.886791    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:05.886943 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:05.886899    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:06.633282 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:06.633241    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 08 07:27:07.380040 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.380005    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"]
Jul 08 07:27:07.380421 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.380249    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" containerID="cri-o://024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5" gracePeriod=30
Jul 08 07:27:07.592940 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.592908    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"]
Jul 08 07:27:07.593219 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.593165    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container" containerID="cri-o://849f58e2ac4655c544963b30969316155fa387ef9c60d9366399c880475162f1" gracePeriod=30
Jul 08 07:27:07.593498 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.593239    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kube-rbac-proxy" containerID="cri-o://e5d0767c6143ff1ea207c20a97d5aeba7aa5ab840de1f6d708b08bb0ef41047c" gracePeriod=30
Jul 08 07:27:07.619925 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.619899    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"]
Jul 08 07:27:07.620219 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.620205    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kube-rbac-proxy"
Jul 08 07:27:07.620296 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.620223    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kube-rbac-proxy"
Jul 08 07:27:07.620296 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.620255    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container"
Jul 08 07:27:07.620296 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.620265    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container"
Jul 08 07:27:07.620447 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.620339    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kserve-container"
Jul 08 07:27:07.620447 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.620353    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aa2e8aa-4b71-4cdf-8d37-3cd0ba5f9206" containerName="kube-rbac-proxy"
Jul 08 07:27:07.623082 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.623063    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.627805 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.627780    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3274f-predictor-serving-cert\""
Jul 08 07:27:07.627932 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.627790    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3274f-kube-rbac-proxy-sar-config\""
Jul 08 07:27:07.637807 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.637750    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"]
Jul 08 07:27:07.713788 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.713764    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.713922 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.713813    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b5afb01-e605-4804-b886-680463ddb105-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.713922 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.713902    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7m4v\" (UniqueName: \"kubernetes.io/projected/3b5afb01-e605-4804-b886-680463ddb105-kube-api-access-s7m4v\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.725006 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.724958    2585 generic.go:356] "Generic (PLEG): container finished" podID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerID="e5d0767c6143ff1ea207c20a97d5aeba7aa5ab840de1f6d708b08bb0ef41047c" exitCode=2
Jul 08 07:27:07.725138 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.725004    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" event={"ID":"1eba7aca-26ed-432d-b809-9a0aaed29bf0","Type":"ContainerDied","Data":"e5d0767c6143ff1ea207c20a97d5aeba7aa5ab840de1f6d708b08bb0ef41047c"}
Jul 08 07:27:07.814439 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.814407    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.814439 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.814447    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b5afb01-e605-4804-b886-680463ddb105-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.814700 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.814482    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7m4v\" (UniqueName: \"kubernetes.io/projected/3b5afb01-e605-4804-b886-680463ddb105-kube-api-access-s7m4v\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.814700 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:27:07.814590    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-3274f-predictor-serving-cert: secret "error-404-isvc-3274f-predictor-serving-cert" not found
Jul 08 07:27:07.814700 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:27:07.814675    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls podName:3b5afb01-e605-4804-b886-680463ddb105 nodeName:}" failed. No retries permitted until 2026-07-08 07:27:08.314653562 +0000 UTC m=+1904.101489856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls") pod "error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" (UID: "3b5afb01-e605-4804-b886-680463ddb105") : secret "error-404-isvc-3274f-predictor-serving-cert" not found
Jul 08 07:27:07.815250 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.815230    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b5afb01-e605-4804-b886-680463ddb105-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:07.822765 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:07.822740    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7m4v\" (UniqueName: \"kubernetes.io/projected/3b5afb01-e605-4804-b886-680463ddb105-kube-api-access-s7m4v\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:08.016087 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.016064    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:27:08.116757 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.116723    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d59aed-5bab-455e-8630-7d19e7af4636-openshift-service-ca-bundle\") pod \"c0d59aed-5bab-455e-8630-7d19e7af4636\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") "
Jul 08 07:27:08.116934 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.116782    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls\") pod \"c0d59aed-5bab-455e-8630-7d19e7af4636\" (UID: \"c0d59aed-5bab-455e-8630-7d19e7af4636\") "
Jul 08 07:27:08.117178 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.117147    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d59aed-5bab-455e-8630-7d19e7af4636-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c0d59aed-5bab-455e-8630-7d19e7af4636" (UID: "c0d59aed-5bab-455e-8630-7d19e7af4636"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:27:08.118814 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.118796    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c0d59aed-5bab-455e-8630-7d19e7af4636" (UID: "c0d59aed-5bab-455e-8630-7d19e7af4636"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:27:08.218018 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.217988    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d59aed-5bab-455e-8630-7d19e7af4636-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:08.218018 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.218019    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0d59aed-5bab-455e-8630-7d19e7af4636-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:08.319212 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.319159    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:08.321549 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.321530    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls\") pod \"error-404-isvc-3274f-predictor-554cc4d85f-w9hq6\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") " pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:08.533359 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.533268    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:08.662507 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.662345    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"]
Jul 08 07:27:08.665344 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:27:08.665320    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5afb01_e605_4804_b886_680463ddb105.slice/crio-f829a12c98b98f37f46e7dbaae5c98a215957dd3348a4cb00c77f250c2637e67 WatchSource:0}: Error finding container f829a12c98b98f37f46e7dbaae5c98a215957dd3348a4cb00c77f250c2637e67: Status 404 returned error can't find the container with id f829a12c98b98f37f46e7dbaae5c98a215957dd3348a4cb00c77f250c2637e67
Jul 08 07:27:08.730995 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.730963    2585 generic.go:356] "Generic (PLEG): container finished" podID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerID="17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6" exitCode=0
Jul 08 07:27:08.731110 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.731058    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" event={"ID":"c0d59aed-5bab-455e-8630-7d19e7af4636","Type":"ContainerDied","Data":"17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6"}
Jul 08 07:27:08.731167 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.731070    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"
Jul 08 07:27:08.731167 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.731122    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4" event={"ID":"c0d59aed-5bab-455e-8630-7d19e7af4636","Type":"ContainerDied","Data":"93c60a171e1210d63d07b9c95bc64aef6dd0f9e6c489808f4dda9aafc9934ce0"}
Jul 08 07:27:08.731167 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.731153    2585 scope.go:117] "RemoveContainer" containerID="17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6"
Jul 08 07:27:08.733117 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.733054    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" event={"ID":"3b5afb01-e605-4804-b886-680463ddb105","Type":"ContainerStarted","Data":"f829a12c98b98f37f46e7dbaae5c98a215957dd3348a4cb00c77f250c2637e67"}
Jul 08 07:27:08.742082 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.742062    2585 scope.go:117] "RemoveContainer" containerID="17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6"
Jul 08 07:27:08.742394 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:27:08.742355    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6\": container with ID starting with 17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6 not found: ID does not exist" containerID="17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6"
Jul 08 07:27:08.742509 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.742400    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6"} err="failed to get container status \"17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6\": rpc error: code = NotFound desc = could not find container \"17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6\": container with ID starting with 17871ac26891380dd247cb96b5632bc948a09bf9d54d0ae4f930c1f45376f0e6 not found: ID does not exist"
Jul 08 07:27:08.758194 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.758168    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"]
Jul 08 07:27:08.762105 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.762085    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f938-77484598f8-rppn4"]
Jul 08 07:27:08.949486 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:08.949457    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" path="/var/lib/kubelet/pods/c0d59aed-5bab-455e-8630-7d19e7af4636/volumes"
Jul 08 07:27:09.739848 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:09.739812    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" event={"ID":"3b5afb01-e605-4804-b886-680463ddb105","Type":"ContainerStarted","Data":"49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317"}
Jul 08 07:27:09.740418 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:09.739873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" event={"ID":"3b5afb01-e605-4804-b886-680463ddb105","Type":"ContainerStarted","Data":"dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020"}
Jul 08 07:27:09.740418 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:09.739957    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:09.758643 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:09.758593    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podStartSLOduration=2.7585785830000003 podStartE2EDuration="2.758578583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:27:09.75846123 +0000 UTC m=+1905.545297595" watchObservedRunningTime="2026-07-08 07:27:09.758578583 +0000 UTC m=+1905.545414890"
Jul 08 07:27:10.744955 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.744927    2585 generic.go:356] "Generic (PLEG): container finished" podID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerID="849f58e2ac4655c544963b30969316155fa387ef9c60d9366399c880475162f1" exitCode=0
Jul 08 07:27:10.745335 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.744956    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" event={"ID":"1eba7aca-26ed-432d-b809-9a0aaed29bf0","Type":"ContainerDied","Data":"849f58e2ac4655c544963b30969316155fa387ef9c60d9366399c880475162f1"}
Jul 08 07:27:10.745335 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.745033    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf" event={"ID":"1eba7aca-26ed-432d-b809-9a0aaed29bf0","Type":"ContainerDied","Data":"27310753170fca9990d78ab4a2cddef6052288a033ab8fc1fd8231a574cd7615"}
Jul 08 07:27:10.745335 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.745069    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27310753170fca9990d78ab4a2cddef6052288a033ab8fc1fd8231a574cd7615"
Jul 08 07:27:10.745335 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.745294    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:10.746710 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.746687    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 08 07:27:10.748043 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.748028    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:27:10.839853 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.839768    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b26xn\" (UniqueName: \"kubernetes.io/projected/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-api-access-b26xn\") pod \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") "
Jul 08 07:27:10.839853 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.839814    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-rbac-proxy-sar-config\") pod \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") "
Jul 08 07:27:10.839853 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.839844    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls\") pod \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\" (UID: \"1eba7aca-26ed-432d-b809-9a0aaed29bf0\") "
Jul 08 07:27:10.840207 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.840183    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1eba7aca-26ed-432d-b809-9a0aaed29bf0" (UID: "1eba7aca-26ed-432d-b809-9a0aaed29bf0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:27:10.841882 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.841858    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-api-access-b26xn" (OuterVolumeSpecName: "kube-api-access-b26xn") pod "1eba7aca-26ed-432d-b809-9a0aaed29bf0" (UID: "1eba7aca-26ed-432d-b809-9a0aaed29bf0"). InnerVolumeSpecName "kube-api-access-b26xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:27:10.842105 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.841937    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1eba7aca-26ed-432d-b809-9a0aaed29bf0" (UID: "1eba7aca-26ed-432d-b809-9a0aaed29bf0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:27:10.941019 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.940988    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b26xn\" (UniqueName: \"kubernetes.io/projected/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-api-access-b26xn\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:10.941019 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.941014    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1eba7aca-26ed-432d-b809-9a0aaed29bf0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:10.941019 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.941026    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1eba7aca-26ed-432d-b809-9a0aaed29bf0-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:10.989478 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:10.989444    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:11.748822 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:11.748786    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"
Jul 08 07:27:11.749213 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:11.749056    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 08 07:27:11.765637 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:11.765608    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"]
Jul 08 07:27:11.768530 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:11.768510    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3d727-predictor-58657f85cc-bp9xf"]
Jul 08 07:27:12.949989 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:12.949946    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" path="/var/lib/kubelet/pods/1eba7aca-26ed-432d-b809-9a0aaed29bf0/volumes"
Jul 08 07:27:15.989397 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:15.989359    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:16.632649 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:16.632608    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 08 07:27:16.754013 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:16.753985    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:16.754481 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:16.754452    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 08 07:27:20.989461 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:20.989415    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:20.990034 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:20.989513    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:27:24.895025 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:24.894997    2585 scope.go:117] "RemoveContainer" containerID="e5d0767c6143ff1ea207c20a97d5aeba7aa5ab840de1f6d708b08bb0ef41047c"
Jul 08 07:27:24.902774 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:24.902749    2585 scope.go:117] "RemoveContainer" containerID="849f58e2ac4655c544963b30969316155fa387ef9c60d9366399c880475162f1"
Jul 08 07:27:25.989795 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:25.989759    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:26.633769 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:26.633732    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:27:26.754691 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:26.754651    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 08 07:27:30.989217 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:30.989180    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:35.990220 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:35.990176    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:36.755402 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:36.755361    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 08 07:27:37.523170 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.523144    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:27:37.643325 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.643222    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fd6661e-00cc-44aa-9516-9f2055ea572c-openshift-service-ca-bundle\") pod \"2fd6661e-00cc-44aa-9516-9f2055ea572c\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") "
Jul 08 07:27:37.643325 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.643292    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls\") pod \"2fd6661e-00cc-44aa-9516-9f2055ea572c\" (UID: \"2fd6661e-00cc-44aa-9516-9f2055ea572c\") "
Jul 08 07:27:37.643699 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.643670    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fd6661e-00cc-44aa-9516-9f2055ea572c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "2fd6661e-00cc-44aa-9516-9f2055ea572c" (UID: "2fd6661e-00cc-44aa-9516-9f2055ea572c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:27:37.645278 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.645256    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2fd6661e-00cc-44aa-9516-9f2055ea572c" (UID: "2fd6661e-00cc-44aa-9516-9f2055ea572c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:27:37.744475 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.744443    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fd6661e-00cc-44aa-9516-9f2055ea572c-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:37.744475 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.744473    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fd6661e-00cc-44aa-9516-9f2055ea572c-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:37.842037 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.842005    2585 generic.go:356] "Generic (PLEG): container finished" podID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerID="024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5" exitCode=0
Jul 08 07:27:37.842217 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.842045    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" event={"ID":"2fd6661e-00cc-44aa-9516-9f2055ea572c","Type":"ContainerDied","Data":"024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5"}
Jul 08 07:27:37.842217 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.842049    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"
Jul 08 07:27:37.842217 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.842086    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5" event={"ID":"2fd6661e-00cc-44aa-9516-9f2055ea572c","Type":"ContainerDied","Data":"f2346c181cc3d1cb6bbdb77cd4cc4dff09310c63e8bdaef3151713281e512ecc"}
Jul 08 07:27:37.842217 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.842109    2585 scope.go:117] "RemoveContainer" containerID="024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5"
Jul 08 07:27:37.849993 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.849974    2585 scope.go:117] "RemoveContainer" containerID="024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5"
Jul 08 07:27:37.850237 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:27:37.850219    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5\": container with ID starting with 024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5 not found: ID does not exist" containerID="024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5"
Jul 08 07:27:37.850287 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.850244    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5"} err="failed to get container status \"024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5\": rpc error: code = NotFound desc = could not find container \"024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5\": container with ID starting with 024c089de82212bd8fdd7422f109980509e47d9da6a17510860d4fedd42389c5 not found: ID does not exist"
Jul 08 07:27:37.862792 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.862769    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"]
Jul 08 07:27:37.866087 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:37.866066    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3d727-6dd4fd7669-crrj5"]
Jul 08 07:27:38.100965 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.100931    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"]
Jul 08 07:27:38.101319 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101306    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727"
Jul 08 07:27:38.101365 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101323    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727"
Jul 08 07:27:38.101365 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101344    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container"
Jul 08 07:27:38.101365 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101353    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container"
Jul 08 07:27:38.101460 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101371    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kube-rbac-proxy"
Jul 08 07:27:38.101460 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101381    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kube-rbac-proxy"
Jul 08 07:27:38.101460 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101389    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938"
Jul 08 07:27:38.101460 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101397    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938"
Jul 08 07:27:38.101460 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101453    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kube-rbac-proxy"
Jul 08 07:27:38.101636 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101465    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d59aed-5bab-455e-8630-7d19e7af4636" containerName="ensemble-graph-1f938"
Jul 08 07:27:38.101636 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101475    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" containerName="sequence-graph-3d727"
Jul 08 07:27:38.101636 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.101485    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eba7aca-26ed-432d-b809-9a0aaed29bf0" containerName="kserve-container"
Jul 08 07:27:38.105797 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.105779    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.108472 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.108450    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-71ab7-kube-rbac-proxy-sar-config\""
Jul 08 07:27:38.108472 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.108466    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-71ab7-serving-cert\""
Jul 08 07:27:38.111642 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.111617    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"]
Jul 08 07:27:38.248323 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.248281    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731ca061-314c-4faf-b913-7318c5a4aa2f-openshift-service-ca-bundle\") pod \"splitter-graph-71ab7-7cd696848d-j79lk\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") " pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.248499 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.248342    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/731ca061-314c-4faf-b913-7318c5a4aa2f-proxy-tls\") pod \"splitter-graph-71ab7-7cd696848d-j79lk\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") " pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.349540 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.349509    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731ca061-314c-4faf-b913-7318c5a4aa2f-openshift-service-ca-bundle\") pod \"splitter-graph-71ab7-7cd696848d-j79lk\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") " pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.349729 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.349550    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/731ca061-314c-4faf-b913-7318c5a4aa2f-proxy-tls\") pod \"splitter-graph-71ab7-7cd696848d-j79lk\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") " pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.350130 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.350109    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731ca061-314c-4faf-b913-7318c5a4aa2f-openshift-service-ca-bundle\") pod \"splitter-graph-71ab7-7cd696848d-j79lk\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") " pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.351910 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.351859    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/731ca061-314c-4faf-b913-7318c5a4aa2f-proxy-tls\") pod \"splitter-graph-71ab7-7cd696848d-j79lk\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") " pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.415648 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.415613    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.542178 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.541996    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"]
Jul 08 07:27:38.544882 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:27:38.544855    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731ca061_314c_4faf_b913_7318c5a4aa2f.slice/crio-1bff7c6b8b66141a5749f6ff812cdd6d243ab50f727a9fb1b10a74b15ba61473 WatchSource:0}: Error finding container 1bff7c6b8b66141a5749f6ff812cdd6d243ab50f727a9fb1b10a74b15ba61473: Status 404 returned error can't find the container with id 1bff7c6b8b66141a5749f6ff812cdd6d243ab50f727a9fb1b10a74b15ba61473
Jul 08 07:27:38.847000 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.846967    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" event={"ID":"731ca061-314c-4faf-b913-7318c5a4aa2f","Type":"ContainerStarted","Data":"d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed"}
Jul 08 07:27:38.847186 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.847022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" event={"ID":"731ca061-314c-4faf-b913-7318c5a4aa2f","Type":"ContainerStarted","Data":"1bff7c6b8b66141a5749f6ff812cdd6d243ab50f727a9fb1b10a74b15ba61473"}
Jul 08 07:27:38.847186 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.847059    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:38.864319 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.864224    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podStartSLOduration=0.864207899 podStartE2EDuration="864.207899ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:27:38.862927261 +0000 UTC m=+1934.649763591" watchObservedRunningTime="2026-07-08 07:27:38.864207899 +0000 UTC m=+1934.651044240"
Jul 08 07:27:38.949349 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:38.949317    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fd6661e-00cc-44aa-9516-9f2055ea572c" path="/var/lib/kubelet/pods/2fd6661e-00cc-44aa-9516-9f2055ea572c/volumes"
Jul 08 07:27:44.858708 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:44.858679    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:27:46.755139 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:46.755103    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 08 07:27:48.182000 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.181911    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"]
Jul 08 07:27:48.182475 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.182154    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" containerID="cri-o://d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed" gracePeriod=30
Jul 08 07:27:48.412025 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.411992    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"]
Jul 08 07:27:48.412256 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.412214    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container" containerID="cri-o://7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7" gracePeriod=30
Jul 08 07:27:48.412324 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.412275    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kube-rbac-proxy" containerID="cri-o://c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1" gracePeriod=30
Jul 08 07:27:48.414785 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.414641    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"]
Jul 08 07:27:48.419335 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.419316    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.421925 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.421904    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9de39-predictor-serving-cert\""
Jul 08 07:27:48.422013 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.421924    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9de39-kube-rbac-proxy-sar-config\""
Jul 08 07:27:48.426247 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.426213    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"]
Jul 08 07:27:48.534532 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.534500    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.534688 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.534589    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-proxy-tls\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.534688 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.534627    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggtg\" (UniqueName: \"kubernetes.io/projected/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-api-access-6ggtg\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.635347 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.635290    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.635347 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.635348    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-proxy-tls\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.635644 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.635370    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggtg\" (UniqueName: \"kubernetes.io/projected/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-api-access-6ggtg\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.636006 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.635987    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.637980 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.637953    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-proxy-tls\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.644450 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.644431    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggtg\" (UniqueName: \"kubernetes.io/projected/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-api-access-6ggtg\") pod \"error-404-isvc-9de39-predictor-6c54447ff5-5mqrh\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") " pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.735495 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.735460    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:48.865725 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.865601    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"]
Jul 08 07:27:48.868387 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:27:48.868357    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a8f50f5_efd3_4dfc_80b7_157faad9a32a.slice/crio-c02f4ceccf9f7e50da8457350c270878f04e045e63ba27e5f084f6ccfef0ec5c WatchSource:0}: Error finding container c02f4ceccf9f7e50da8457350c270878f04e045e63ba27e5f084f6ccfef0ec5c: Status 404 returned error can't find the container with id c02f4ceccf9f7e50da8457350c270878f04e045e63ba27e5f084f6ccfef0ec5c
Jul 08 07:27:48.885288 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.885264    2585 generic.go:356] "Generic (PLEG): container finished" podID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerID="c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1" exitCode=2
Jul 08 07:27:48.885394 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.885334    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" event={"ID":"73d21b91-ffca-4f69-9520-f46c9f9f2259","Type":"ContainerDied","Data":"c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1"}
Jul 08 07:27:48.886503 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:48.886480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" event={"ID":"6a8f50f5-efd3-4dfc-80b7-157faad9a32a","Type":"ContainerStarted","Data":"c02f4ceccf9f7e50da8457350c270878f04e045e63ba27e5f084f6ccfef0ec5c"}
Jul 08 07:27:49.856113 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:49.856073    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:49.891879 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:49.891846    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" event={"ID":"6a8f50f5-efd3-4dfc-80b7-157faad9a32a","Type":"ContainerStarted","Data":"8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a"}
Jul 08 07:27:49.892059 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:49.891911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" event={"ID":"6a8f50f5-efd3-4dfc-80b7-157faad9a32a","Type":"ContainerStarted","Data":"0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9"}
Jul 08 07:27:49.892059 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:49.891992    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:49.910256 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:49.910206    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podStartSLOduration=1.910190889 podStartE2EDuration="1.910190889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:27:49.908386208 +0000 UTC m=+1945.695222523" watchObservedRunningTime="2026-07-08 07:27:49.910190889 +0000 UTC m=+1945.697027205"
Jul 08 07:27:50.896174 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:50.896141    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:50.897324 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:50.897298    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 08 07:27:51.654254 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.654224    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:27:51.764636 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.764602    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm9sc\" (UniqueName: \"kubernetes.io/projected/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-api-access-jm9sc\") pod \"73d21b91-ffca-4f69-9520-f46c9f9f2259\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") "
Jul 08 07:27:51.764831 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.764684    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls\") pod \"73d21b91-ffca-4f69-9520-f46c9f9f2259\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") "
Jul 08 07:27:51.764831 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.764715    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-rbac-proxy-sar-config\") pod \"73d21b91-ffca-4f69-9520-f46c9f9f2259\" (UID: \"73d21b91-ffca-4f69-9520-f46c9f9f2259\") "
Jul 08 07:27:51.765080 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.765057    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "73d21b91-ffca-4f69-9520-f46c9f9f2259" (UID: "73d21b91-ffca-4f69-9520-f46c9f9f2259"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:27:51.766692 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.766669    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-api-access-jm9sc" (OuterVolumeSpecName: "kube-api-access-jm9sc") pod "73d21b91-ffca-4f69-9520-f46c9f9f2259" (UID: "73d21b91-ffca-4f69-9520-f46c9f9f2259"). InnerVolumeSpecName "kube-api-access-jm9sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:27:51.766759 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.766693    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "73d21b91-ffca-4f69-9520-f46c9f9f2259" (UID: "73d21b91-ffca-4f69-9520-f46c9f9f2259"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:27:51.865350 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.865316    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm9sc\" (UniqueName: \"kubernetes.io/projected/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-api-access-jm9sc\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:51.865350 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.865344    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73d21b91-ffca-4f69-9520-f46c9f9f2259-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:51.865350 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.865355    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73d21b91-ffca-4f69-9520-f46c9f9f2259-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:27:51.901251 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.901217    2585 generic.go:356] "Generic (PLEG): container finished" podID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerID="7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7" exitCode=0
Jul 08 07:27:51.901720 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.901267    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"
Jul 08 07:27:51.901720 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.901298    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" event={"ID":"73d21b91-ffca-4f69-9520-f46c9f9f2259","Type":"ContainerDied","Data":"7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7"}
Jul 08 07:27:51.901720 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.901375    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" event={"ID":"73d21b91-ffca-4f69-9520-f46c9f9f2259","Type":"ContainerDied","Data":"aeb9ce507d8127a82267dd500d3fdc9ce32a760a26da452dd09e1b2bce3ac0f4"}
Jul 08 07:27:51.901720 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.901416    2585 scope.go:117] "RemoveContainer" containerID="c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1"
Jul 08 07:27:51.901981 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.901954    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 08 07:27:51.909710 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.909694    2585 scope.go:117] "RemoveContainer" containerID="7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7"
Jul 08 07:27:51.916605 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.916588    2585 scope.go:117] "RemoveContainer" containerID="c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1"
Jul 08 07:27:51.916828 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:27:51.916813    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1\": container with ID starting with c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1 not found: ID does not exist" containerID="c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1"
Jul 08 07:27:51.916871 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.916835    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1"} err="failed to get container status \"c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1\": rpc error: code = NotFound desc = could not find container \"c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1\": container with ID starting with c0f16edd0dd1e3d5d5db78564ebd2c614a8bcf5edab10c6847006b14eb39a3f1 not found: ID does not exist"
Jul 08 07:27:51.916871 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.916859    2585 scope.go:117] "RemoveContainer" containerID="7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7"
Jul 08 07:27:51.917104 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:27:51.917085    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7\": container with ID starting with 7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7 not found: ID does not exist" containerID="7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7"
Jul 08 07:27:51.917159 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.917108    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7"} err="failed to get container status \"7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7\": rpc error: code = NotFound desc = could not find container \"7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7\": container with ID starting with 7f0977f89c064c35b258e4c2a188af93e6326a5e534861045e6837989938f4c7 not found: ID does not exist"
Jul 08 07:27:51.921929 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.921906    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"]
Jul 08 07:27:51.925263 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:51.925247    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw"]
Jul 08 07:27:52.628186 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:52.628133    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-71ab7-predictor-868947f459-fxsgw" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": context deadline exceeded"
Jul 08 07:27:52.949761 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:52.949726    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" path="/var/lib/kubelet/pods/73d21b91-ffca-4f69-9520-f46c9f9f2259/volumes"
Jul 08 07:27:54.856701 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:54.856665    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:56.755377 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:56.755348    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:27:56.906273 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:56.906248    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:27:56.906694 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:56.906665    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 08 07:27:59.856171 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:59.856130    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:27:59.856645 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:27:59.856236    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:28:04.856447 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:04.856405    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:28:06.906753 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:06.906719    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 08 07:28:07.610615 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610580    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"]
Jul 08 07:28:07.610855 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610844    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container"
Jul 08 07:28:07.610894 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610857    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container"
Jul 08 07:28:07.610894 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610870    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kube-rbac-proxy"
Jul 08 07:28:07.610894 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610876    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kube-rbac-proxy"
Jul 08 07:28:07.610998 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610927    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kube-rbac-proxy"
Jul 08 07:28:07.610998 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.610936    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="73d21b91-ffca-4f69-9520-f46c9f9f2259" containerName="kserve-container"
Jul 08 07:28:07.613618 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.613603    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.616341 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.616308    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-3274f-serving-cert\""
Jul 08 07:28:07.616341 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.616329    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-3274f-kube-rbac-proxy-sar-config\""
Jul 08 07:28:07.622101 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.622078    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"]
Jul 08 07:28:07.785972 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.785938    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51c33ba9-9fa2-4212-80a3-6c4d805a0170-proxy-tls\") pod \"switch-graph-3274f-78b9ff9b6-mr4hr\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") " pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.786142 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.786015    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51c33ba9-9fa2-4212-80a3-6c4d805a0170-openshift-service-ca-bundle\") pod \"switch-graph-3274f-78b9ff9b6-mr4hr\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") " pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.887442 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.887364    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51c33ba9-9fa2-4212-80a3-6c4d805a0170-openshift-service-ca-bundle\") pod \"switch-graph-3274f-78b9ff9b6-mr4hr\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") " pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.887442 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.887414    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51c33ba9-9fa2-4212-80a3-6c4d805a0170-proxy-tls\") pod \"switch-graph-3274f-78b9ff9b6-mr4hr\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") " pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.888181 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.888153    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51c33ba9-9fa2-4212-80a3-6c4d805a0170-openshift-service-ca-bundle\") pod \"switch-graph-3274f-78b9ff9b6-mr4hr\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") " pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.889769 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.889750    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51c33ba9-9fa2-4212-80a3-6c4d805a0170-proxy-tls\") pod \"switch-graph-3274f-78b9ff9b6-mr4hr\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") " pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:07.924033 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:07.923993    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:08.053927 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:08.053844    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"]
Jul 08 07:28:08.056393 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:28:08.056370    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c33ba9_9fa2_4212_80a3_6c4d805a0170.slice/crio-199e7a8ea98b407196f62ea789e14ac3bfc5d1ffdd0aea3c018d60bdbeef4067 WatchSource:0}: Error finding container 199e7a8ea98b407196f62ea789e14ac3bfc5d1ffdd0aea3c018d60bdbeef4067: Status 404 returned error can't find the container with id 199e7a8ea98b407196f62ea789e14ac3bfc5d1ffdd0aea3c018d60bdbeef4067
Jul 08 07:28:08.964391 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:08.964360    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" event={"ID":"51c33ba9-9fa2-4212-80a3-6c4d805a0170","Type":"ContainerStarted","Data":"583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85"}
Jul 08 07:28:08.964825 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:08.964404    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" event={"ID":"51c33ba9-9fa2-4212-80a3-6c4d805a0170","Type":"ContainerStarted","Data":"199e7a8ea98b407196f62ea789e14ac3bfc5d1ffdd0aea3c018d60bdbeef4067"}
Jul 08 07:28:08.964825 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:08.964458    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:08.980660 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:08.980617    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podStartSLOduration=1.980602266 podStartE2EDuration="1.980602266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:28:08.979637662 +0000 UTC m=+1964.766473972" watchObservedRunningTime="2026-07-08 07:28:08.980602266 +0000 UTC m=+1964.767438582"
Jul 08 07:28:09.856698 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:09.856658    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:28:14.855867 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:14.855823    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:28:14.974473 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:14.974444    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:28:16.906773 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:16.906683    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 08 07:28:18.322215 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.322192    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:28:18.366341 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.366312    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/731ca061-314c-4faf-b913-7318c5a4aa2f-proxy-tls\") pod \"731ca061-314c-4faf-b913-7318c5a4aa2f\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") "
Jul 08 07:28:18.366507 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.366352    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731ca061-314c-4faf-b913-7318c5a4aa2f-openshift-service-ca-bundle\") pod \"731ca061-314c-4faf-b913-7318c5a4aa2f\" (UID: \"731ca061-314c-4faf-b913-7318c5a4aa2f\") "
Jul 08 07:28:18.366785 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.366759    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731ca061-314c-4faf-b913-7318c5a4aa2f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "731ca061-314c-4faf-b913-7318c5a4aa2f" (UID: "731ca061-314c-4faf-b913-7318c5a4aa2f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:28:18.368376 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.368349    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731ca061-314c-4faf-b913-7318c5a4aa2f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "731ca061-314c-4faf-b913-7318c5a4aa2f" (UID: "731ca061-314c-4faf-b913-7318c5a4aa2f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:28:18.466976 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.466940    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/731ca061-314c-4faf-b913-7318c5a4aa2f-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:28:18.466976 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:18.466969    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731ca061-314c-4faf-b913-7318c5a4aa2f-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:28:19.000164 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.000132    2585 generic.go:356] "Generic (PLEG): container finished" podID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerID="d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed" exitCode=0
Jul 08 07:28:19.000327 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.000174    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"
Jul 08 07:28:19.000327 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.000184    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" event={"ID":"731ca061-314c-4faf-b913-7318c5a4aa2f","Type":"ContainerDied","Data":"d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed"}
Jul 08 07:28:19.000327 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.000231    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk" event={"ID":"731ca061-314c-4faf-b913-7318c5a4aa2f","Type":"ContainerDied","Data":"1bff7c6b8b66141a5749f6ff812cdd6d243ab50f727a9fb1b10a74b15ba61473"}
Jul 08 07:28:19.000327 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.000254    2585 scope.go:117] "RemoveContainer" containerID="d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed"
Jul 08 07:28:19.008185 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.008165    2585 scope.go:117] "RemoveContainer" containerID="d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed"
Jul 08 07:28:19.008442 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:28:19.008421    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed\": container with ID starting with d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed not found: ID does not exist" containerID="d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed"
Jul 08 07:28:19.008488 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.008450    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed"} err="failed to get container status \"d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed\": rpc error: code = NotFound desc = could not find container \"d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed\": container with ID starting with d9c548c3b8a7ea68936b8b02ab8df1c893bbd7c2dc61d022bb516753ff8fe8ed not found: ID does not exist"
Jul 08 07:28:19.016878 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.016852    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"]
Jul 08 07:28:19.020611 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:19.020589    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-71ab7-7cd696848d-j79lk"]
Jul 08 07:28:20.949673 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:20.949640    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" path="/var/lib/kubelet/pods/731ca061-314c-4faf-b913-7318c5a4aa2f/volumes"
Jul 08 07:28:26.907550 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:26.907511    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 08 07:28:36.907765 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:36.907734    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:28:48.401501 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.401463    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"]
Jul 08 07:28:48.401953 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.401809    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7"
Jul 08 07:28:48.401953 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.401822    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7"
Jul 08 07:28:48.401953 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.401865    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="731ca061-314c-4faf-b913-7318c5a4aa2f" containerName="splitter-graph-71ab7"
Jul 08 07:28:48.404633 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.404618    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.407484 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.407462    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-9de39-serving-cert\""
Jul 08 07:28:48.407617 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.407470    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-9de39-kube-rbac-proxy-sar-config\""
Jul 08 07:28:48.413206 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.413184    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"]
Jul 08 07:28:48.481017 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.480984    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc103665-6bb6-4354-b003-5a77d82ed2bc-openshift-service-ca-bundle\") pod \"splitter-graph-9de39-68884c5cfc-t9prr\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") " pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.481245 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.481039    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc103665-6bb6-4354-b003-5a77d82ed2bc-proxy-tls\") pod \"splitter-graph-9de39-68884c5cfc-t9prr\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") " pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.581527 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.581487    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc103665-6bb6-4354-b003-5a77d82ed2bc-openshift-service-ca-bundle\") pod \"splitter-graph-9de39-68884c5cfc-t9prr\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") " pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.581725 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.581555    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc103665-6bb6-4354-b003-5a77d82ed2bc-proxy-tls\") pod \"splitter-graph-9de39-68884c5cfc-t9prr\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") " pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.582103 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.582083    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc103665-6bb6-4354-b003-5a77d82ed2bc-openshift-service-ca-bundle\") pod \"splitter-graph-9de39-68884c5cfc-t9prr\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") " pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.583979 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.583957    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc103665-6bb6-4354-b003-5a77d82ed2bc-proxy-tls\") pod \"splitter-graph-9de39-68884c5cfc-t9prr\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") " pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.716964 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.716934    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:48.839717 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:48.839693    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"]
Jul 08 07:28:48.841867 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:28:48.841841    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc103665_6bb6_4354_b003_5a77d82ed2bc.slice/crio-05689a636547499bfc07011c11eeffc8ad4d7887a3709bf80eba0aa471bf1cac WatchSource:0}: Error finding container 05689a636547499bfc07011c11eeffc8ad4d7887a3709bf80eba0aa471bf1cac: Status 404 returned error can't find the container with id 05689a636547499bfc07011c11eeffc8ad4d7887a3709bf80eba0aa471bf1cac
Jul 08 07:28:49.105693 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:49.105603    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" event={"ID":"bc103665-6bb6-4354-b003-5a77d82ed2bc","Type":"ContainerStarted","Data":"1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d"}
Jul 08 07:28:49.105693 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:49.105666    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" event={"ID":"bc103665-6bb6-4354-b003-5a77d82ed2bc","Type":"ContainerStarted","Data":"05689a636547499bfc07011c11eeffc8ad4d7887a3709bf80eba0aa471bf1cac"}
Jul 08 07:28:49.105874 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:49.105705    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:28:49.123381 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:49.123337    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podStartSLOduration=1.123321086 podStartE2EDuration="1.123321086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:28:49.121715292 +0000 UTC m=+2004.908551608" watchObservedRunningTime="2026-07-08 07:28:49.123321086 +0000 UTC m=+2004.910157402"
Jul 08 07:28:55.115549 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:28:55.115521    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:37:03.046043 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.046011    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"]
Jul 08 07:37:03.048441 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.046202    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" containerID="cri-o://1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d" gracePeriod=30
Jul 08 07:37:03.199156 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.199128    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"]
Jul 08 07:37:03.199364 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.199342    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container" containerID="cri-o://0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9" gracePeriod=30
Jul 08 07:37:03.199436 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.199390    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kube-rbac-proxy" containerID="cri-o://8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a" gracePeriod=30
Jul 08 07:37:03.834379 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.834332    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerID="8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a" exitCode=2
Jul 08 07:37:03.834610 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:03.834403    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" event={"ID":"6a8f50f5-efd3-4dfc-80b7-157faad9a32a","Type":"ContainerDied","Data":"8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a"}
Jul 08 07:37:05.114227 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:05.114184    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:37:06.444919 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.444898    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:37:06.519154 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.519066    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-rbac-proxy-sar-config\") pod \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") "
Jul 08 07:37:06.519154 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.519118    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-proxy-tls\") pod \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") "
Jul 08 07:37:06.519154 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.519137    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ggtg\" (UniqueName: \"kubernetes.io/projected/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-api-access-6ggtg\") pod \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\" (UID: \"6a8f50f5-efd3-4dfc-80b7-157faad9a32a\") "
Jul 08 07:37:06.519452 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.519427    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6a8f50f5-efd3-4dfc-80b7-157faad9a32a" (UID: "6a8f50f5-efd3-4dfc-80b7-157faad9a32a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:37:06.521139 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.521117    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6a8f50f5-efd3-4dfc-80b7-157faad9a32a" (UID: "6a8f50f5-efd3-4dfc-80b7-157faad9a32a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:37:06.521261 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.521237    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-api-access-6ggtg" (OuterVolumeSpecName: "kube-api-access-6ggtg") pod "6a8f50f5-efd3-4dfc-80b7-157faad9a32a" (UID: "6a8f50f5-efd3-4dfc-80b7-157faad9a32a"). InnerVolumeSpecName "kube-api-access-6ggtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:37:06.620587 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.620521    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:37:06.620587 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.620577    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:37:06.620587 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.620589    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ggtg\" (UniqueName: \"kubernetes.io/projected/6a8f50f5-efd3-4dfc-80b7-157faad9a32a-kube-api-access-6ggtg\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:37:06.846943 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.846849    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerID="0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9" exitCode=0
Jul 08 07:37:06.846943 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.846904    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"
Jul 08 07:37:06.847158 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.846906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" event={"ID":"6a8f50f5-efd3-4dfc-80b7-157faad9a32a","Type":"ContainerDied","Data":"0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9"}
Jul 08 07:37:06.847158 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.847037    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh" event={"ID":"6a8f50f5-efd3-4dfc-80b7-157faad9a32a","Type":"ContainerDied","Data":"c02f4ceccf9f7e50da8457350c270878f04e045e63ba27e5f084f6ccfef0ec5c"}
Jul 08 07:37:06.847158 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.847070    2585 scope.go:117] "RemoveContainer" containerID="8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a"
Jul 08 07:37:06.855400 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.855385    2585 scope.go:117] "RemoveContainer" containerID="0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9"
Jul 08 07:37:06.862415 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.862400    2585 scope.go:117] "RemoveContainer" containerID="8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a"
Jul 08 07:37:06.862675 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:37:06.862658    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a\": container with ID starting with 8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a not found: ID does not exist" containerID="8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a"
Jul 08 07:37:06.862731 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.862684    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a"} err="failed to get container status \"8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a\": rpc error: code = NotFound desc = could not find container \"8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a\": container with ID starting with 8aecea683323736700bbb647f8fba0f743417f3bb32a6b9a521b054f38b0b63a not found: ID does not exist"
Jul 08 07:37:06.862731 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.862700    2585 scope.go:117] "RemoveContainer" containerID="0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9"
Jul 08 07:37:06.862919 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:37:06.862900    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9\": container with ID starting with 0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9 not found: ID does not exist" containerID="0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9"
Jul 08 07:37:06.862962 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.862926    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9"} err="failed to get container status \"0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9\": rpc error: code = NotFound desc = could not find container \"0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9\": container with ID starting with 0d4bcdcd3bf99c8ff71bd8149d12ba30ec03325c6808a52dec00892f2c49acc9 not found: ID does not exist"
Jul 08 07:37:06.867754 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.867735    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"]
Jul 08 07:37:06.873496 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.873467    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9de39-predictor-6c54447ff5-5mqrh"]
Jul 08 07:37:06.949708 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:06.949682    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" path="/var/lib/kubelet/pods/6a8f50f5-efd3-4dfc-80b7-157faad9a32a/volumes"
Jul 08 07:37:10.114472 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:10.114430    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:37:15.115033 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:15.114992    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:37:15.115490 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:15.115102    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:37:20.114082 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:20.113995    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:37:25.115104 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:25.115060    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:37:30.114714 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:30.114663    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:37:33.200847 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.200819    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:37:33.233174 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.233146    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc103665-6bb6-4354-b003-5a77d82ed2bc-openshift-service-ca-bundle\") pod \"bc103665-6bb6-4354-b003-5a77d82ed2bc\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") "
Jul 08 07:37:33.233174 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.233177    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc103665-6bb6-4354-b003-5a77d82ed2bc-proxy-tls\") pod \"bc103665-6bb6-4354-b003-5a77d82ed2bc\" (UID: \"bc103665-6bb6-4354-b003-5a77d82ed2bc\") "
Jul 08 07:37:33.233493 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.233472    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc103665-6bb6-4354-b003-5a77d82ed2bc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "bc103665-6bb6-4354-b003-5a77d82ed2bc" (UID: "bc103665-6bb6-4354-b003-5a77d82ed2bc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:37:33.235054 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.235020    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc103665-6bb6-4354-b003-5a77d82ed2bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bc103665-6bb6-4354-b003-5a77d82ed2bc" (UID: "bc103665-6bb6-4354-b003-5a77d82ed2bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:37:33.334283 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.334198    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc103665-6bb6-4354-b003-5a77d82ed2bc-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:37:33.334283 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.334228    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc103665-6bb6-4354-b003-5a77d82ed2bc-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:37:33.944240 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.944209    2585 generic.go:356] "Generic (PLEG): container finished" podID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerID="1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d" exitCode=137
Jul 08 07:37:33.944409 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.944253    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"
Jul 08 07:37:33.944409 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.944292    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" event={"ID":"bc103665-6bb6-4354-b003-5a77d82ed2bc","Type":"ContainerDied","Data":"1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d"}
Jul 08 07:37:33.944409 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.944363    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr" event={"ID":"bc103665-6bb6-4354-b003-5a77d82ed2bc","Type":"ContainerDied","Data":"05689a636547499bfc07011c11eeffc8ad4d7887a3709bf80eba0aa471bf1cac"}
Jul 08 07:37:33.944409 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.944385    2585 scope.go:117] "RemoveContainer" containerID="1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d"
Jul 08 07:37:33.952575 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.952536    2585 scope.go:117] "RemoveContainer" containerID="1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d"
Jul 08 07:37:33.952899 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:37:33.952881    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d\": container with ID starting with 1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d not found: ID does not exist" containerID="1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d"
Jul 08 07:37:33.952971 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.952910    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d"} err="failed to get container status \"1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d\": rpc error: code = NotFound desc = could not find container \"1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d\": container with ID starting with 1cd6342ba827d533a858f54a5a5e9e6a9ebbac4b802b85fe2ca58337e2c8704d not found: ID does not exist"
Jul 08 07:37:33.967922 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.967891    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"]
Jul 08 07:37:33.973608 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:33.973586    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-9de39-68884c5cfc-t9prr"]
Jul 08 07:37:34.949620 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:37:34.949590    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" path="/var/lib/kubelet/pods/bc103665-6bb6-4354-b003-5a77d82ed2bc/volumes"
Jul 08 07:44:26.902084 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:26.902049    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"]
Jul 08 07:44:26.904443 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:26.902264    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" containerID="cri-o://583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85" gracePeriod=30
Jul 08 07:44:27.273697 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:27.273661    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"]
Jul 08 07:44:27.273949 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:27.273898    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container" containerID="cri-o://49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317" gracePeriod=30
Jul 08 07:44:27.274092 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:27.273954    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kube-rbac-proxy" containerID="cri-o://dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020" gracePeriod=30
Jul 08 07:44:28.346381 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:28.346342    2585 generic.go:356] "Generic (PLEG): container finished" podID="3b5afb01-e605-4804-b886-680463ddb105" containerID="dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020" exitCode=2
Jul 08 07:44:28.346772 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:28.346412    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" event={"ID":"3b5afb01-e605-4804-b886-680463ddb105","Type":"ContainerDied","Data":"dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020"}
Jul 08 07:44:29.972150 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:29.972114    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:44:30.118037 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.118009    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:44:30.259254 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.259225    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7m4v\" (UniqueName: \"kubernetes.io/projected/3b5afb01-e605-4804-b886-680463ddb105-kube-api-access-s7m4v\") pod \"3b5afb01-e605-4804-b886-680463ddb105\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") "
Jul 08 07:44:30.259429 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.259275    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b5afb01-e605-4804-b886-680463ddb105-kube-rbac-proxy-sar-config\") pod \"3b5afb01-e605-4804-b886-680463ddb105\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") "
Jul 08 07:44:30.259429 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.259342    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls\") pod \"3b5afb01-e605-4804-b886-680463ddb105\" (UID: \"3b5afb01-e605-4804-b886-680463ddb105\") "
Jul 08 07:44:30.259700 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.259667    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b5afb01-e605-4804-b886-680463ddb105-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3b5afb01-e605-4804-b886-680463ddb105" (UID: "3b5afb01-e605-4804-b886-680463ddb105"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:44:30.261305 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.261266    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3b5afb01-e605-4804-b886-680463ddb105" (UID: "3b5afb01-e605-4804-b886-680463ddb105"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:44:30.261415 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.261362    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5afb01-e605-4804-b886-680463ddb105-kube-api-access-s7m4v" (OuterVolumeSpecName: "kube-api-access-s7m4v") pod "3b5afb01-e605-4804-b886-680463ddb105" (UID: "3b5afb01-e605-4804-b886-680463ddb105"). InnerVolumeSpecName "kube-api-access-s7m4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:44:30.354920 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.354883    2585 generic.go:356] "Generic (PLEG): container finished" podID="3b5afb01-e605-4804-b886-680463ddb105" containerID="49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317" exitCode=0
Jul 08 07:44:30.355077 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.354935    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"
Jul 08 07:44:30.355077 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.354971    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" event={"ID":"3b5afb01-e605-4804-b886-680463ddb105","Type":"ContainerDied","Data":"49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317"}
Jul 08 07:44:30.355077 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.355053    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6" event={"ID":"3b5afb01-e605-4804-b886-680463ddb105","Type":"ContainerDied","Data":"f829a12c98b98f37f46e7dbaae5c98a215957dd3348a4cb00c77f250c2637e67"}
Jul 08 07:44:30.355193 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.355079    2585 scope.go:117] "RemoveContainer" containerID="dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020"
Jul 08 07:44:30.360050 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.360029    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7m4v\" (UniqueName: \"kubernetes.io/projected/3b5afb01-e605-4804-b886-680463ddb105-kube-api-access-s7m4v\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:44:30.360150 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.360054    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b5afb01-e605-4804-b886-680463ddb105-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:44:30.360150 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.360070    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b5afb01-e605-4804-b886-680463ddb105-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:44:30.363219 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.363167    2585 scope.go:117] "RemoveContainer" containerID="49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317"
Jul 08 07:44:30.372178 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.372162    2585 scope.go:117] "RemoveContainer" containerID="dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020"
Jul 08 07:44:30.372418 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:44:30.372399    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020\": container with ID starting with dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020 not found: ID does not exist" containerID="dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020"
Jul 08 07:44:30.372469 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.372426    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020"} err="failed to get container status \"dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020\": rpc error: code = NotFound desc = could not find container \"dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020\": container with ID starting with dd6f0541b52cf0df19db9e9be65442d964a004a3c2d0ec92d843ba7e7f3e8020 not found: ID does not exist"
Jul 08 07:44:30.372469 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.372444    2585 scope.go:117] "RemoveContainer" containerID="49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317"
Jul 08 07:44:30.372731 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:44:30.372715    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317\": container with ID starting with 49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317 not found: ID does not exist" containerID="49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317"
Jul 08 07:44:30.372804 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.372734    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317"} err="failed to get container status \"49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317\": rpc error: code = NotFound desc = could not find container \"49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317\": container with ID starting with 49424fb2e8fc7ef1710db10f510fbb286d911fc253be3854efeb81fb68922317 not found: ID does not exist"
Jul 08 07:44:30.380002 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.379979    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"]
Jul 08 07:44:30.387019 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.386999    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3274f-predictor-554cc4d85f-w9hq6"]
Jul 08 07:44:30.949494 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:30.949462    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b5afb01-e605-4804-b886-680463ddb105" path="/var/lib/kubelet/pods/3b5afb01-e605-4804-b886-680463ddb105/volumes"
Jul 08 07:44:34.971906 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:34.971865    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:44:39.972643 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:39.972604    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:44:39.973131 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:39.972734    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:44:44.327595 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:44.327547    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:44.972705 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:44.972665    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:44:45.250584 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:45.250483    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:46.118003 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:46.117975    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:47.060170 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:47.060140    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:47.994005 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:47.993976    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:48.835367 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:48.835339    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:49.696707 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:49.696680    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:49.972766 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:49.972674    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:44:50.581423 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:50.581397    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:51.495871 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:51.495841    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:52.469177 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:52.469143    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:53.410495 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:53.410470    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:54.370133 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:54.370105    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:54.971633 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:54.971597    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 07:44:55.327345 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:55.327240    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:56.246492 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:56.246453    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3274f-78b9ff9b6-mr4hr_51c33ba9-9fa2-4212-80a3-6c4d805a0170/switch-graph-3274f/0.log"
Jul 08 07:44:57.042918 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.042895    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:44:57.137903 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.137867    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51c33ba9-9fa2-4212-80a3-6c4d805a0170-openshift-service-ca-bundle\") pod \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") "
Jul 08 07:44:57.138100 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.137933    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51c33ba9-9fa2-4212-80a3-6c4d805a0170-proxy-tls\") pod \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\" (UID: \"51c33ba9-9fa2-4212-80a3-6c4d805a0170\") "
Jul 08 07:44:57.138236 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.138211    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51c33ba9-9fa2-4212-80a3-6c4d805a0170-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "51c33ba9-9fa2-4212-80a3-6c4d805a0170" (UID: "51c33ba9-9fa2-4212-80a3-6c4d805a0170"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:44:57.139894 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.139873    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51c33ba9-9fa2-4212-80a3-6c4d805a0170-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "51c33ba9-9fa2-4212-80a3-6c4d805a0170" (UID: "51c33ba9-9fa2-4212-80a3-6c4d805a0170"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:44:57.238465 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.238434    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51c33ba9-9fa2-4212-80a3-6c4d805a0170-openshift-service-ca-bundle\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:44:57.238465 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.238460    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51c33ba9-9fa2-4212-80a3-6c4d805a0170-proxy-tls\") on node \"ip-10-0-142-155.ec2.internal\" DevicePath \"\""
Jul 08 07:44:57.444886 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.444851    2585 generic.go:356] "Generic (PLEG): container finished" podID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerID="583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85" exitCode=0
Jul 08 07:44:57.444886 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.444890    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"
Jul 08 07:44:57.445413 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.444922    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" event={"ID":"51c33ba9-9fa2-4212-80a3-6c4d805a0170","Type":"ContainerDied","Data":"583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85"}
Jul 08 07:44:57.445413 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.444972    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr" event={"ID":"51c33ba9-9fa2-4212-80a3-6c4d805a0170","Type":"ContainerDied","Data":"199e7a8ea98b407196f62ea789e14ac3bfc5d1ffdd0aea3c018d60bdbeef4067"}
Jul 08 07:44:57.445413 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.445006    2585 scope.go:117] "RemoveContainer" containerID="583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85"
Jul 08 07:44:57.453526 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.453508    2585 scope.go:117] "RemoveContainer" containerID="583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85"
Jul 08 07:44:57.453802 ip-10-0-142-155 kubenswrapper[2585]: E0708 07:44:57.453780    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85\": container with ID starting with 583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85 not found: ID does not exist" containerID="583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85"
Jul 08 07:44:57.453876 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.453809    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85"} err="failed to get container status \"583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85\": rpc error: code = NotFound desc = could not find container \"583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85\": container with ID starting with 583fe13a985173ebcccd27185448475228f83d299de1428f4c68877eafbd0d85 not found: ID does not exist"
Jul 08 07:44:57.472797 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.472769    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"]
Jul 08 07:44:57.475695 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:57.475674    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3274f-78b9ff9b6-mr4hr"]
Jul 08 07:44:58.949221 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:44:58.949184    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" path="/var/lib/kubelet/pods/51c33ba9-9fa2-4212-80a3-6c4d805a0170/volumes"
Jul 08 07:45:01.728499 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:01.728471    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xpq58_fcd81370-ee72-4e93-9089-44790495c67b/global-pull-secret-syncer/0.log"
Jul 08 07:45:01.830050 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:01.830022    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qpzwf_91e000e0-f8fc-4045-84cb-ae1a8b8cae39/konnectivity-agent/0.log"
Jul 08 07:45:01.925874 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:01.925847    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-155.ec2.internal_8aeda92cfb515398ae0d19a52ccec463/haproxy/0.log"
Jul 08 07:45:06.902751 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:06.902727    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4gkjf_01e62250-3ed2-411e-b681-4fc78f9728f3/node-exporter/0.log"
Jul 08 07:45:06.929055 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:06.929023    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4gkjf_01e62250-3ed2-411e-b681-4fc78f9728f3/kube-rbac-proxy/0.log"
Jul 08 07:45:06.961441 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:06.961413    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4gkjf_01e62250-3ed2-411e-b681-4fc78f9728f3/init-textfile/0.log"
Jul 08 07:45:09.854775 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.854739    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"]
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.854965    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.854976    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.854985    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.854991    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.854997    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855002    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855018    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855023    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855029    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kube-rbac-proxy"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855034    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kube-rbac-proxy"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855043    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kube-rbac-proxy"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855048    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kube-rbac-proxy"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855085    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kube-rbac-proxy"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855092    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc103665-6bb6-4354-b003-5a77d82ed2bc" containerName="splitter-graph-9de39"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855099    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="51c33ba9-9fa2-4212-80a3-6c4d805a0170" containerName="switch-graph-3274f"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855104    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kube-rbac-proxy"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855109    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5afb01-e605-4804-b886-680463ddb105" containerName="kserve-container"
Jul 08 07:45:09.855161 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.855116    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a8f50f5-efd3-4dfc-80b7-157faad9a32a" containerName="kserve-container"
Jul 08 07:45:09.857060 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.857044    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:09.868217 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.868198    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qznxx\"/\"openshift-service-ca.crt\""
Jul 08 07:45:09.868313 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.868204    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qznxx\"/\"default-dockercfg-tdfcs\""
Jul 08 07:45:09.868313 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.868242    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qznxx\"/\"kube-root-ca.crt\""
Jul 08 07:45:09.871848 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.871829    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"]
Jul 08 07:45:09.933319 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.933292    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-lib-modules\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:09.933319 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.933324    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-sys\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:09.933511 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.933340    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-proc\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:09.933511 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.933358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcv2z\" (UniqueName: \"kubernetes.io/projected/b10b1a28-c327-4c77-9c45-cdda6447863d-kube-api-access-dcv2z\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:09.933511 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:09.933411    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-podres\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034302 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034252    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-proc\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034302 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034307    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcv2z\" (UniqueName: \"kubernetes.io/projected/b10b1a28-c327-4c77-9c45-cdda6447863d-kube-api-access-dcv2z\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034541 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034389    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-proc\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034541 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034416    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-podres\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034541 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034500    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-lib-modules\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034541 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034519    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-sys\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034736 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034597    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-sys\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034736 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034612    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-podres\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.034736 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.034660    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b10b1a28-c327-4c77-9c45-cdda6447863d-lib-modules\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.045660 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.045635    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcv2z\" (UniqueName: \"kubernetes.io/projected/b10b1a28-c327-4c77-9c45-cdda6447863d-kube-api-access-dcv2z\") pod \"perf-node-gather-daemonset-z6w76\" (UID: \"b10b1a28-c327-4c77-9c45-cdda6447863d\") " pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.166838 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.166739    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.296930 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.296900    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"]
Jul 08 07:45:10.299955 ip-10-0-142-155 kubenswrapper[2585]: W0708 07:45:10.299930    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb10b1a28_c327_4c77_9c45_cdda6447863d.slice/crio-3a6a0272d226a0d8debc087593be7dcf43512955e79c41e85bda4dc55f47e178 WatchSource:0}: Error finding container 3a6a0272d226a0d8debc087593be7dcf43512955e79c41e85bda4dc55f47e178: Status 404 returned error can't find the container with id 3a6a0272d226a0d8debc087593be7dcf43512955e79c41e85bda4dc55f47e178
Jul 08 07:45:10.490177 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.490148    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76" event={"ID":"b10b1a28-c327-4c77-9c45-cdda6447863d","Type":"ContainerStarted","Data":"4027b2cb4f6f990c18949745dd7317ed52157efe0b0674399925f8910f3edf4c"}
Jul 08 07:45:10.490302 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.490196    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76" event={"ID":"b10b1a28-c327-4c77-9c45-cdda6447863d","Type":"ContainerStarted","Data":"3a6a0272d226a0d8debc087593be7dcf43512955e79c41e85bda4dc55f47e178"}
Jul 08 07:45:10.490302 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.490227    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:10.522260 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:10.522220    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76" podStartSLOduration=1.52220424 podStartE2EDuration="1.52220424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 07:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:45:10.522079143 +0000 UTC m=+2986.308915459" watchObservedRunningTime="2026-07-08 07:45:10.52220424 +0000 UTC m=+2986.309040556"
Jul 08 07:45:12.099767 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:12.099740    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lqgk_828fffaa-c27d-4036-bb01-bd2b404fd358/dns/0.log"
Jul 08 07:45:12.142202 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:12.142176    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lqgk_828fffaa-c27d-4036-bb01-bd2b404fd358/kube-rbac-proxy/0.log"
Jul 08 07:45:12.306814 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:12.306786    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mczhb_a3d2cf38-1712-4fcc-b70b-564b399cfa2b/dns-node-resolver/0.log"
Jul 08 07:45:12.823323 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:12.823286    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c9bb86f5c-2gs7f_24b0353c-e521-4edc-815e-ea9136aa6541/registry/0.log"
Jul 08 07:45:12.892302 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:12.892267    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k74tn_9cb20c91-2a09-456c-ae6e-d43b64f3aa93/node-ca/0.log"
Jul 08 07:45:14.386387 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:14.386358    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ll8tp_19ab5bed-e37e-413e-a574-003de2a693b9/serve-healthcheck-canary/0.log"
Jul 08 07:45:15.116290 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:15.116258    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcfnd_737866f6-e412-4126-916b-ff22685fc6b1/kube-rbac-proxy/0.log"
Jul 08 07:45:15.138066 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:15.138042    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcfnd_737866f6-e412-4126-916b-ff22685fc6b1/exporter/0.log"
Jul 08 07:45:15.167052 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:15.167029    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcfnd_737866f6-e412-4126-916b-ff22685fc6b1/extractor/0.log"
Jul 08 07:45:16.502869 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:16.502844    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qznxx/perf-node-gather-daemonset-z6w76"
Jul 08 07:45:17.875467 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:17.875436    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-tjfgk_ba5bcee4-10dc-4620-8f16-83882623f7c9/server/0.log"
Jul 08 07:45:18.099890 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:18.099864    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-4w4dx_0d316631-a099-45f4-9741-2888ee314ea5/seaweedfs/0.log"
Jul 08 07:45:24.099747 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.099718    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/kube-multus-additional-cni-plugins/0.log"
Jul 08 07:45:24.130706 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.130679    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/egress-router-binary-copy/0.log"
Jul 08 07:45:24.153580 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.153543    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/cni-plugins/0.log"
Jul 08 07:45:24.175732 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.175707    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/bond-cni-plugin/0.log"
Jul 08 07:45:24.201774 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.201754    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/routeoverride-cni/0.log"
Jul 08 07:45:24.225406 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.225378    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/whereabouts-cni-bincopy/0.log"
Jul 08 07:45:24.250122 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.250104    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lsgx2_9e84c25b-8b4c-49b5-87a1-4b1f1f10d42c/whereabouts-cni/0.log"
Jul 08 07:45:24.716120 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.716092    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lkzph_8520e981-4ad3-4258-bd34-b9eb7bdcecd4/kube-multus/0.log"
Jul 08 07:45:24.767174 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.767141    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mlrcx_6de506fd-0787-407c-97e3-349973298b4f/network-metrics-daemon/0.log"
Jul 08 07:45:24.790366 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:24.790343    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mlrcx_6de506fd-0787-407c-97e3-349973298b4f/kube-rbac-proxy/0.log"
Jul 08 07:45:26.283046 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.283016    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/ovn-controller/0.log"
Jul 08 07:45:26.302597 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.302557    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/ovn-acl-logging/0.log"
Jul 08 07:45:26.329219 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.329196    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/kube-rbac-proxy-node/0.log"
Jul 08 07:45:26.353178 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.353143    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/kube-rbac-proxy-ovn-metrics/0.log"
Jul 08 07:45:26.377839 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.377812    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/northd/0.log"
Jul 08 07:45:26.401471 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.401450    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/nbdb/0.log"
Jul 08 07:45:26.429855 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.429823    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/sbdb/0.log"
Jul 08 07:45:26.478184 ip-10-0-142-155 kubenswrapper[2585]: I0708 07:45:26.478155    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4jqd_eb5ab310-b866-4cda-86fc-689295b20301/ovnkube-controller/0.log"