Jul 08 16:32:59.289106 ip-10-0-132-182 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.780919    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787031    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787043    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787047    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787050    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787053    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787055    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787058    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:32:59.889845 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787060    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787063    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787066    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787069    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787071    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787074    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787076    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787079    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787082    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787086    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787088    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787091    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787094    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787097    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787100    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787102    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787107    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787111    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787121    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787125    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:33:00.164857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787127    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:32:59.864972 ip-10-0-132-182 systemd[1]: Started Kubernetes Kubelet.
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787130    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787133    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787136    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787139    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787141    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787144    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787147    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787149    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787152    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787154    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787157    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787159    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787163    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787165    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787168    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787171    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787173    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787176    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787179    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:33:00.179807 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787181    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787184    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787186    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787189    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787192    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787197    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787201    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787204    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787207    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787210    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787212    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787215    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787217    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787220    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787224    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787226    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787229    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787231    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787234    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:33:00.180629 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787237    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787239    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787242    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787244    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787248    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787252    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787255    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787258    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787261    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787263    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787266    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787269    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787272    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787274    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787277    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787281    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787284    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787287    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787290    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787292    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:33:00.181465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787295    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787297    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787300    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787302    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787305    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787307    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787310    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787313    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787315    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787318    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787321    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787324    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.787326    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788039    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788046    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788049    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788053    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788056    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788058    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788061    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:33:00.182297 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788065    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788068    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788071    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788073    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788076    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788078    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788081    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788085    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788087    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788090    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788092    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788095    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788098    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788101    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788103    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788107    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788111    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788114    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788117    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:33:00.182915 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788120    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788124    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788128    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788133    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788136    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788139    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788141    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788144    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788147    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788150    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788153    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788156    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788158    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788161    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788164    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788166    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788169    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788172    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788174    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:33:00.183516 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788177    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788180    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788183    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788185    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788187    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788190    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788193    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788197    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788200    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788203    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788206    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788209    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788211    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788214    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788217    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788220    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788224    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788227    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788230    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788232    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:33:00.184330 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788235    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788237    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788240    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788242    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788245    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788247    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788250    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788253    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788255    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788258    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788260    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788263    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788266    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788268    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788271    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788274    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788277    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788279    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788282    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:33:00.184956 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788284    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788287    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788290    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788292    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788295    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788297    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788300    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788302    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788305    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788308    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788311    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788313    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788316    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788319    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788321    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788688    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788692    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788695    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788698    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:33:00.185561 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788700    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788703    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788705    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788708    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788711    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788713    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788718    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788722    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788725    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788728    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788731    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788734    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788737    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788740    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788742    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788745    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788747    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788750    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788752    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788755    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:33:00.186134 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788757    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788760    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788764    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788766    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788769    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788772    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788774    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788777    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788779    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788782    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788785    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788789    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788791    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788794    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788796    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788799    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788801    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788804    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788806    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788809    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:33:00.186736 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788811    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788814    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788817    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788841    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788852    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788860    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788865    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788868    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788871    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788873    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788876    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788878    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788881    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788883    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788888    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788891    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788893    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788896    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788898    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788901    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:33:00.187320 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788903    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788906    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788908    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788913    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788915    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788918    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788920    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788923    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788926    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788929    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788931    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788933    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788936    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788939    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788942    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788945    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788947    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788950    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788952    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788954    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:33:00.187941 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788957    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788959    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788961    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788964    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788966    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788969    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788975    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788978    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788982    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788984    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788987    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788990    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788993    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788995    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.788997    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789067    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789074    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789081    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789085    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789089    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789093    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 16:33:00.188501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789097    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789101    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789104    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789107    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789110    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789113    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789117    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789120    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789123    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789126    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789129    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789132    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789136    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789138    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789141    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789144    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789147    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789152    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789155    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789158    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789161    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789164    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789167    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789170    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 16:33:00.189116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789173    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789176    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789181    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789184    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789187    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789190    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789193    2562 flags.go:64] FLAG: --enable-server="true"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789196    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789200    2562 flags.go:64] FLAG: --event-burst="100"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789203    2562 flags.go:64] FLAG: --event-qps="50"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789206    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789209    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789212    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789215    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789218    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789222    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789225    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789228    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789231    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789234    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789237    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789240    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789243    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789246    2562 flags.go:64] FLAG: --feature-gates=""
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789249    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 16:33:00.189768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789253    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789256    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789260    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789262    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789265    2562 flags.go:64] FLAG: --help="false"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789268    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789271    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789274    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789277    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789281    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789285    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789287    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789290    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789293    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789296    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789298    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789301    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789304    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789307    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789310    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789313    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789316    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789319    2562 flags.go:64] FLAG: --lock-file=""
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789322    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 16:33:00.190505 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789338    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789341    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789347    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789350    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789354    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789356    2562 flags.go:64] FLAG: --logging-format="text"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789359    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789362    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789366    2562 flags.go:64] FLAG: --manifest-url=""
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789369    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789373    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789376    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789380    2562 flags.go:64] FLAG: --max-pods="110"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789383    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789386    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789389    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789392    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789396    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789399    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789402    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789409    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789412    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789415    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 16:33:00.191154 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789418    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789421    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789426    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789429    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789432    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789435    2562 flags.go:64] FLAG: --port="10250"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789438    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789441    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0597e310162f678d5"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789444    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789448    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789451    2562 flags.go:64] FLAG: --register-node="true"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789454    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789457    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789460    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789463    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789466    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789469    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789472    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789477    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789479    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789482    2562 flags.go:64] FLAG: --runonce="false"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789485    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789488    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789491    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789494    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 16:33:00.191794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789496    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789501    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789504    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789507    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789509    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789512    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789515    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789518    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789521    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789523    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789526    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789532    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789534    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789537    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789540    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789543    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789546    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789550    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789553    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789556    2562 flags.go:64] FLAG: --v="2"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789560    2562 flags.go:64] FLAG: --version="false"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789564    2562 flags.go:64] FLAG: --vmodule=""
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789567    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789571    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789688    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:33:00.192735 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789693    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789696    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789700    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789703    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789707    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789711    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789714    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789717    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789722    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789724    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789727    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789730    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789733    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789735    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789738    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789740    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789743    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789747    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789750    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:33:00.193437 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789753    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789756    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789759    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789761    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789764    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789766    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789769    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789772    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789774    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789777    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789779    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789782    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789785    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789788    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789791    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789793    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789797    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789800    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789803    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789806    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:33:00.326883 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789808    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789812    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789814    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789817    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789835    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789837    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789840    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789842    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789846    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789848    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789851    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789853    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789856    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789859    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789861    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789864    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789867    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789869    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789871    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789875    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:33:00.327543 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789877    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789880    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789882    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789885    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789888    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789891    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789894    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789896    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789899    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789901    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789904    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789906    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789909    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789912    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789915    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789918    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789920    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789923    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789925    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789928    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:33:00.328098 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789930    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789933    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789935    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789938    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789941    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789943    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789945    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789948    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789951    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789953    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789956    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789959    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789962    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789964    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789967    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789969    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789972    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789976    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:33:00.328655 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.789979    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.789987    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.797359    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.797374    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.799394    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.799406    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799465    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799470    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799473    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799476    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799480    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799483    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799485    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799488    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799491    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:33:00.329169 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799494    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799496    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799499    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799502    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799505    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799508    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799510    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799513    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799516    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799521    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799524    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799527    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799530    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799533    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799535    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799538    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799540    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799544    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799548    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:33:00.329654 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799551    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799555    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799558    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799561    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799564    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799567    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799570    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799573    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799576    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799579    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799581    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799584    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799587    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799589    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799592    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799594    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799597    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799600    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799602    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:33:00.330170 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799605    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799607    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799610    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799614    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799617    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799620    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799623    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799625    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799628    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799631    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799634    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799636    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799639    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799642    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799644    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799647    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799649    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799652    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799654    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799657    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:33:00.330699 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799660    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799664    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799667    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799669    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799672    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799675    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799678    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799682    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799685    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799689    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799692    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799694    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799697    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799700    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799703    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799707    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799710    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799713    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799716    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799719    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:33:00.331270 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799721    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799724    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799726    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799729    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799731    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799734    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799736    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799739    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799741    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799744    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799746    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799749    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.799754    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799882    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:33:00.331857 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799890    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799893    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799896    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799898    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799901    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799904    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799907    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799910    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799912    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799915    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799918    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799920    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799923    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799928    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799931    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799934    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799937    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799939    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799942    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:33:00.332251 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799945    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799948    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799950    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799954    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799958    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799960    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799963    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799966    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799969    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799971    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799974    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799976    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799978    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799984    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799987    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799990    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799993    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799995    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.799998    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:33:00.332814 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800001    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800003    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800006    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800008    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800011    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800013    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800016    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800020    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800022    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800025    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800027    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800030    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800033    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800036    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800039    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800041    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800044    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800046    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800049    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800051    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:33:00.333343 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800053    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800056    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800058    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800061    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800063    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800066    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800069    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800072    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800074    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800077    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800079    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800082    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800084    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800087    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800089    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800092    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800095    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800097    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800099    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800105    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:33:00.333921 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800107    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800109    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800112    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800114    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800117    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800119    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800122    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800125    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800128    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800130    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800132    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800135    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800138    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800140    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800142    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800145    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800147    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800150    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800154    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:33:00.334448 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:32:59.800156    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.800161    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.800272    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.802339    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.805221    2562 server.go:1019] "Starting client certificate rotation"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.805315    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.805346    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.808063    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.810114    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.824215    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.827470    2562 log.go:25] "Validated CRI v1 image API"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.828479    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 16:33:00.334990 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.836447    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b24e54e0-2587-43da-84cb-057c96751601:/dev/nvme0n1p4 d7df1345-4677-4a76-bdab-4ba18ea562fb:/dev/nvme0n1p3]
Jul 08 16:33:00.335326 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.836461    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.841604    2562 manager.go:216] Machine: {Timestamp:2026-07-08 16:32:59.840045367 +0000 UTC m=+0.402022742 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099963 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24db68062f6d9a89478d314993b10c SystemUUID:ec24db68-062f-6d9a-8947-8d314993b10c BootID:74a24367-7565-4b2e-866d-bacdda209b9f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8d:9b:aa:40:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8d:9b:aa:40:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:65:b3:03:fd:87 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.841750    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.841864    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.845264    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.845294    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-182.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.846749    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 16:33:00.335425 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.846801    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.846813    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.846835    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.847650    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.849019    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852172    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852280    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852310    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852335    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852355    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852368    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852379    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852769    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.852784    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.855194    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.857204    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.857239    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858463    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858479    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858488    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858499    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858511    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858520    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858531    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858538    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858548    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858557    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858578    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.858587    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.859501    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.859510    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.862844    2562 server.go:1268] "Started kubelet"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.862912    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.862982    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.863022    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.864720    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.869242    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-182.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.869320    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.869356    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-182.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 16:33:00.335671 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.869434    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.870945    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.871835    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-182.ec2.internal.18c05d9ca88cf72d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-182.ec2.internal,UID:ip-10-0-132-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-182.ec2.internal,},FirstTimestamp:2026-07-08 16:32:59.862808365 +0000 UTC m=+0.424785740,LastTimestamp:2026-07-08 16:32:59.862808365 +0000 UTC m=+0.424785740,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-182.ec2.internal,}"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876033    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876090    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.876102    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876772    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876785    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876803    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876858    2562 factory.go:55] Registering systemd factory
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876873    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876894    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.876902    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.877089    2562 factory.go:153] Registering CRI-O factory
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.877103    2562 factory.go:223] Registration of the crio container factory successfully
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.877152    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.877182    2562 factory.go:103] Registering Raw factory
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.877197    2562 manager.go:1196] Started watching for new ooms in manager
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.877581    2562 manager.go:349] Starting recovery of all containers
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.887190    2562 manager.go:354] Recovery completed
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.887264    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-182.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.887287    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.891638    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893327    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893360    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893369    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893888    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893897    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893913    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.893927    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.894624    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.894643    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.894656    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.336738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.902257    2562 policy_none.go:49] "None policy: Start"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.902271    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.902281    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.903754    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-182.ec2.internal.18c05d9caa5eeecc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-182.ec2.internal,UID:ip-10-0-132-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-182.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-182.ec2.internal,},FirstTimestamp:2026-07-08 16:32:59.893345996 +0000 UTC m=+0.455323375,LastTimestamp:2026-07-08 16:32:59.893345996 +0000 UTC m=+0.455323375,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-182.ec2.internal,}"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.909444    2562 policy_none.go:47] "Start"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.911718    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-182.ec2.internal.18c05d9caa5f36de  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-182.ec2.internal,UID:ip-10-0-132-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-182.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-182.ec2.internal,},FirstTimestamp:2026-07-08 16:32:59.893364446 +0000 UTC m=+0.455341821,LastTimestamp:2026-07-08 16:32:59.893364446 +0000 UTC m=+0.455341821,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-182.ec2.internal,}"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.921912    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-182.ec2.internal.18c05d9caa5f5850  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-182.ec2.internal,UID:ip-10-0-132-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-182.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-182.ec2.internal,},FirstTimestamp:2026-07-08 16:32:59.893373008 +0000 UTC m=+0.455350386,LastTimestamp:2026-07-08 16:32:59.893373008 +0000 UTC m=+0.455350386,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-182.ec2.internal,}"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.922644    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wgbqd"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:32:59.932277    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wgbqd"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.935378    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-182.ec2.internal.18c05d9caa5eeecc\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-182.ec2.internal.18c05d9caa5eeecc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-182.ec2.internal,UID:ip-10-0-132-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-182.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-182.ec2.internal,},FirstTimestamp:2026-07-08 16:32:59.893345996 +0000 UTC m=+0.455323375,LastTimestamp:2026-07-08 16:32:59.894633942 +0000 UTC m=+0.456611317,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-182.ec2.internal,}"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:32:59.976985    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014357    2562 manager.go:337] "Starting Device Plugin manager"
Jul 08 16:33:00.488971 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.014386    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014396    2562 server.go:87] "Starting device plugin registration server"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014633    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014648    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014772    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014859    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.014867    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.015382    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.015435    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.021518    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.022693    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.022733    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.022756    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.022793    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.025571    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.094524    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-182.ec2.internal\" not found" node="ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.115628    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.117308    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.117335    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.117346    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.117369    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.122857    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"]
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.122917    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.123655    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.123707    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.123728    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.125416    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.125535    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.125562    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126109    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126129    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126139    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126111    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126201    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126213    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.126736    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.490380 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.126749    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-182.ec2.internal\": node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.144045    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.144075    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.144766    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.144791    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.144804    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.159462    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-182.ec2.internal\" not found" node="ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.167786    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.171997    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-182.ec2.internal\" not found" node="ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.178265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43f0a89a04d165574d4c36c955b6bc1e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-182.ec2.internal\" (UID: \"43f0a89a04d165574d4c36c955b6bc1e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.178420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/19047cfbf21b92e7ac69508e6d358f35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal\" (UID: \"19047cfbf21b92e7ac69508e6d358f35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.178439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19047cfbf21b92e7ac69508e6d358f35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal\" (UID: \"19047cfbf21b92e7ac69508e6d358f35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.268050    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.278939    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19047cfbf21b92e7ac69508e6d358f35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal\" (UID: \"19047cfbf21b92e7ac69508e6d358f35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.278971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19047cfbf21b92e7ac69508e6d358f35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal\" (UID: \"19047cfbf21b92e7ac69508e6d358f35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.278989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43f0a89a04d165574d4c36c955b6bc1e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-182.ec2.internal\" (UID: \"43f0a89a04d165574d4c36c955b6bc1e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.279005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/19047cfbf21b92e7ac69508e6d358f35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal\" (UID: \"19047cfbf21b92e7ac69508e6d358f35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.279028    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/19047cfbf21b92e7ac69508e6d358f35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal\" (UID: \"19047cfbf21b92e7ac69508e6d358f35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.492453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.279052    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43f0a89a04d165574d4c36c955b6bc1e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-182.ec2.internal\" (UID: \"43f0a89a04d165574d4c36c955b6bc1e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.493030 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.369005    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.493030 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.462995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.493030 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.469757    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.493030 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.476504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.646539 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.569838    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.811445 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.672977    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.811445 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:00.780671    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-182.ec2.internal\" not found"
Jul 08 16:33:00.811445 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.805804    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 16:33:00.811445 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.806006    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.857463    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.877248    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.877251    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.889660    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.892083    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.892320    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.905438    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.916956    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-684jl"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.926426    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-684jl"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.941816    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 16:28:00 +0000 UTC" deadline="2027-12-21 14:42:04.568869418 +0000 UTC"
Jul 08 16:33:00.969474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:00.941856    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12742h9m3.627016758s"
Jul 08 16:33:01.252678 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.221586    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:33:01.365724 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.263007    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:33:01.365724 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:01.356107    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19047cfbf21b92e7ac69508e6d358f35.slice/crio-5fc34b863559ac7ecac932358b7cc2a118d728bcd09f0336b2144203812aac85 WatchSource:0}: Error finding container 5fc34b863559ac7ecac932358b7cc2a118d728bcd09f0336b2144203812aac85: Status 404 returned error can't find the container with id 5fc34b863559ac7ecac932358b7cc2a118d728bcd09f0336b2144203812aac85
Jul 08 16:33:01.365724 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:01.356610    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f0a89a04d165574d4c36c955b6bc1e.slice/crio-a0183438e459d92873c8e3bdb2ce42e52c518860516160d786f1fb1ed9752670 WatchSource:0}: Error finding container a0183438e459d92873c8e3bdb2ce42e52c518860516160d786f1fb1ed9752670: Status 404 returned error can't find the container with id a0183438e459d92873c8e3bdb2ce42e52c518860516160d786f1fb1ed9752670
Jul 08 16:33:01.365724 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.363792    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.853960    2562 apiserver.go:52] "Watching apiserver"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.864340    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.864791    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mc44p","kube-system/konnectivity-agent-89vm6","kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal","openshift-dns/node-resolver-wzp4k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal","openshift-multus/multus-6jjf5","openshift-multus/multus-additional-cni-plugins-sr2lk","openshift-network-operator/iptables-alerter-56rnv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss","openshift-cluster-node-tuning-operator/tuned-7p92d","openshift-image-registry/node-ca-cpggg","openshift-multus/network-metrics-daemon-br9fz","openshift-network-diagnostics/network-check-target-64tq5","openshift-ovn-kubernetes/ovnkube-node-gx2vd"]
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.866442    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.867743    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.869552    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.869559    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.869569    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.870464    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q9dq2\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.870514    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.870570    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4mw5n\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.870703    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.872975    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.874502    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.875451    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.875513    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.875597    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.876011    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.876146    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.876155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tjzlh\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.876196    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.877038    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.877071    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.877562    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:01.877624    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.877854    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q879r\""
Jul 08 16:33:01.913401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.878908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.880281    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.881767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.881884    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z74s8\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.882020    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:01.883254    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883352    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7pjxq\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883583    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883594    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.883631    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.884458    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.884459    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rbv4m\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.884784    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.884821    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.885296    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:01.885368    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.885526    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ksv6h\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.887317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.890105    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.890168    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.890605    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vs8hx\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.891384    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.891446    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.891458    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 16:33:01.917521 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.891602    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-cni-bin\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-multus-certs\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f281f6c-3a96-491b-bb22-f8ae225f25b1-hosts-file\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-cni-multus\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-socket-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2jn5\" (UniqueName: \"kubernetes.io/projected/ec55c32f-1872-4d62-94e5-ff0c736bec34-kube-api-access-n2jn5\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c511300-5a57-43ae-b976-92821d4b9b59-agent-certs\") pod \"konnectivity-agent-89vm6\" (UID: \"1c511300-5a57-43ae-b976-92821d4b9b59\") " pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74fgg\" (UniqueName: \"kubernetes.io/projected/4f281f6c-3a96-491b-bb22-f8ae225f25b1-kube-api-access-74fgg\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-os-release\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-cni-binary-copy\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-k8s-cni-cncf-io\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-conf-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.919376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.893985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-etc-kubernetes\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-system-cni-dir\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-os-release\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqgq4\" (UniqueName: \"kubernetes.io/projected/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-kube-api-access-sqgq4\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-hostroot\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e967c6f7-5197-4648-8ce2-cffcd788026d-kubelet-config\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-registration-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-socket-dir-parent\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-kubelet\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-etc-selinux\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c511300-5a57-43ae-b976-92821d4b9b59-konnectivity-ca\") pod \"konnectivity-agent-89vm6\" (UID: \"1c511300-5a57-43ae-b976-92821d4b9b59\") " pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cni-binary-copy\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.922618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e967c6f7-5197-4648-8ce2-cffcd788026d-dbus\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-device-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-system-cni-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-cni-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-cnibin\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-netns\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.894969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-daemon-config\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.895011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8l6h\" (UniqueName: \"kubernetes.io/projected/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-kube-api-access-k8l6h\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.895045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-sys-fs\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.895108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cnibin\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.895153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.895191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:01.923322 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.895234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f281f6c-3a96-491b-bb22-f8ae225f25b1-tmp-dir\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.929626    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:28:01 +0000 UTC" deadline="2028-01-04 11:25:47.659093323 +0000 UTC"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.929700    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13074h52m45.729401043s"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.979059    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.980871    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.995885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-netns\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.995948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-sys-fs\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.995982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996017    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-netns\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cnibin\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996098    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-sys-fs\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-tuned\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-systemd-units\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-node-log\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.029734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-multus-certs\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996339    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cnibin\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996329    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-multus-certs\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whdm2\" (UniqueName: \"kubernetes.io/projected/9cb737a7-e603-4199-86ec-e44ec1e67392-kube-api-access-whdm2\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed690582-0a9e-4069-892c-78c07ea8066a-serviceca\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-kubelet\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-var-lib-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996505    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-cni-multus\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996590    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-cni-multus\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-socket-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9njf\" (UniqueName: \"kubernetes.io/projected/ed690582-0a9e-4069-892c-78c07ea8066a-kube-api-access-l9njf\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996699    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-socket-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.031571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c511300-5a57-43ae-b976-92821d4b9b59-agent-certs\") pod \"konnectivity-agent-89vm6\" (UID: \"1c511300-5a57-43ae-b976-92821d4b9b59\") " pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-kubernetes\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed690582-0a9e-4069-892c-78c07ea8066a-host\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-log-socket\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovnkube-config\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-system-cni-dir\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.996994    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e967c6f7-5197-4648-8ce2-cffcd788026d-kubelet-config\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997047    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-system-cni-dir\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997061    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e967c6f7-5197-4648-8ce2-cffcd788026d-kubelet-config\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-var-lib-kubelet\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997097    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzhm7\" (UniqueName: \"kubernetes.io/projected/850ce6be-0856-49f4-9b25-cf2a0396d1a5-kube-api-access-hzhm7\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997122    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 16:33:02.032297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-etc-selinux\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/114e39d2-544d-4713-9b42-efe1494a319f-iptables-alerter-script\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysconfig\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-run\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c511300-5a57-43ae-b976-92821d4b9b59-konnectivity-ca\") pod \"konnectivity-agent-89vm6\" (UID: \"1c511300-5a57-43ae-b976-92821d4b9b59\") " pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997271    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-etc-selinux\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e967c6f7-5197-4648-8ce2-cffcd788026d-dbus\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-etc-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-system-cni-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-cni-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:01.997442    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997452    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e967c6f7-5197-4648-8ce2-cffcd788026d-dbus\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997496    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-cnibin\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.033475 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:01.997512    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:33:02.49749074 +0000 UTC m=+3.059468109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-cnibin\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997544    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-system-cni-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-daemon-config\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997558    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-cni-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8l6h\" (UniqueName: \"kubernetes.io/projected/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-kube-api-access-k8l6h\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27hc8\" (UniqueName: \"kubernetes.io/projected/cce8909d-08fb-45cb-a131-394cd09252a9-kube-api-access-27hc8\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovn-node-metrics-cert\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f281f6c-3a96-491b-bb22-f8ae225f25b1-tmp-dir\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-cni-bin\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997802    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114e39d2-544d-4713-9b42-efe1494a319f-host-slash\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997839    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-cni-bin\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cb737a7-e603-4199-86ec-e44ec1e67392-tmp\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-run-netns\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.034346 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997936    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-systemd\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-cni-bin\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.997999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-env-overrides\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f281f6c-3a96-491b-bb22-f8ae225f25b1-hosts-file\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998059    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jn5\" (UniqueName: \"kubernetes.io/projected/ec55c32f-1872-4d62-94e5-ff0c736bec34-kube-api-access-n2jn5\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998063    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c511300-5a57-43ae-b976-92821d4b9b59-konnectivity-ca\") pod \"konnectivity-agent-89vm6\" (UID: \"1c511300-5a57-43ae-b976-92821d4b9b59\") " pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75mp8\" (UniqueName: \"kubernetes.io/projected/114e39d2-544d-4713-9b42-efe1494a319f-kube-api-access-75mp8\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998106    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f281f6c-3a96-491b-bb22-f8ae225f25b1-tmp-dir\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysctl-d\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998141    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-daemon-config\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998143    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f281f6c-3a96-491b-bb22-f8ae225f25b1-hosts-file\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysctl-conf\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-ovn\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-cni-netd\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.035797 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74fgg\" (UniqueName: \"kubernetes.io/projected/4f281f6c-3a96-491b-bb22-f8ae225f25b1-kube-api-access-74fgg\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-os-release\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-cni-binary-copy\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-k8s-cni-cncf-io\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-conf-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998586    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-run-k8s-cni-cncf-io\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998697    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-os-release\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-etc-kubernetes\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-lib-modules\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-os-release\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqgq4\" (UniqueName: \"kubernetes.io/projected/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-kube-api-access-sqgq4\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998834    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.036648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-hostroot\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-registration-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998915    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-systemd\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998926    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-etc-kubernetes\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-sys\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998980    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.998983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-socket-dir-parent\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999063    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-os-release\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999053    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-cni-binary-copy\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-kubelet\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999098    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-host-var-lib-kubelet\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-modprobe-d\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999130    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-conf-dir\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.037539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999155    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-multus-socket-dir-parent\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-host\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999178    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-registration-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999183    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-hostroot\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-slash\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999212    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cni-binary-copy\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-device-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovnkube-script-lib\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999369    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec55c32f-1872-4d62-94e5-ff0c736bec34-device-dir\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:01.999856    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-cni-binary-copy\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.002390    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c511300-5a57-43ae-b976-92821d4b9b59-agent-certs\") pod \"konnectivity-agent-89vm6\" (UID: \"1c511300-5a57-43ae-b976-92821d4b9b59\") " pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.010315    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2jn5\" (UniqueName: \"kubernetes.io/projected/ec55c32f-1872-4d62-94e5-ff0c736bec34-kube-api-access-n2jn5\") pod \"aws-ebs-csi-driver-node-swsss\" (UID: \"ec55c32f-1872-4d62-94e5-ff0c736bec34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.010788    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8l6h\" (UniqueName: \"kubernetes.io/projected/1d57f4c7-60a7-47f1-8edb-3bcd4661c518-kube-api-access-k8l6h\") pod \"multus-6jjf5\" (UID: \"1d57f4c7-60a7-47f1-8edb-3bcd4661c518\") " pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.039401 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.010939    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74fgg\" (UniqueName: \"kubernetes.io/projected/4f281f6c-3a96-491b-bb22-f8ae225f25b1-kube-api-access-74fgg\") pod \"node-resolver-wzp4k\" (UID: \"4f281f6c-3a96-491b-bb22-f8ae225f25b1\") " pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.040075 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.011333    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqgq4\" (UniqueName: \"kubernetes.io/projected/6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f-kube-api-access-sqgq4\") pod \"multus-additional-cni-plugins-sr2lk\" (UID: \"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f\") " pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.040075 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.028595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal" event={"ID":"43f0a89a04d165574d4c36c955b6bc1e","Type":"ContainerStarted","Data":"a0183438e459d92873c8e3bdb2ce42e52c518860516160d786f1fb1ed9752670"}
Jul 08 16:33:02.040075 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.030927    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal" event={"ID":"19047cfbf21b92e7ac69508e6d358f35","Type":"ContainerStarted","Data":"5fc34b863559ac7ecac932358b7cc2a118d728bcd09f0336b2144203812aac85"}
Jul 08 16:33:02.100467 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovnkube-script-lib\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.100467 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:02.100618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.100618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100506    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-tuned\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.100618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100565    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.100749 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-systemd-units\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.100749 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-node-log\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.100874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whdm2\" (UniqueName: \"kubernetes.io/projected/9cb737a7-e603-4199-86ec-e44ec1e67392-kube-api-access-whdm2\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.100874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed690582-0a9e-4069-892c-78c07ea8066a-serviceca\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.100874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-kubelet\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.100874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-var-lib-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9njf\" (UniqueName: \"kubernetes.io/projected/ed690582-0a9e-4069-892c-78c07ea8066a-kube-api-access-l9njf\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100907    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-node-log\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-kubernetes\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100967    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-var-lib-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100819    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-systemd-units\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100991    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-kubernetes\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.100995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed690582-0a9e-4069-892c-78c07ea8066a-host\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101052    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-log-socket\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovnkube-config\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-var-lib-kubelet\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101116    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed690582-0a9e-4069-892c-78c07ea8066a-host\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101053    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-kubelet\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzhm7\" (UniqueName: \"kubernetes.io/projected/850ce6be-0856-49f4-9b25-cf2a0396d1a5-kube-api-access-hzhm7\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/114e39d2-544d-4713-9b42-efe1494a319f-iptables-alerter-script\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysconfig\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101227    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovnkube-script-lib\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-run\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101252    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-log-socket\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101311    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-var-lib-kubelet\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101406    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysconfig\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.101431    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101445    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed690582-0a9e-4069-892c-78c07ea8066a-serviceca\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101463    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-run\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.101667 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.101501    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:02.601469895 +0000 UTC m=+3.163447260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-etc-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27hc8\" (UniqueName: \"kubernetes.io/projected/cce8909d-08fb-45cb-a131-394cd09252a9-kube-api-access-27hc8\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovn-node-metrics-cert\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114e39d2-544d-4713-9b42-efe1494a319f-host-slash\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cb737a7-e603-4199-86ec-e44ec1e67392-tmp\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101694    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-etc-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-run-netns\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-systemd\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101874    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovnkube-config\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101913    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-cni-bin\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-env-overrides\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101957    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-run-netns\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101983    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/114e39d2-544d-4713-9b42-efe1494a319f-iptables-alerter-script\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.102514 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75mp8\" (UniqueName: \"kubernetes.io/projected/114e39d2-544d-4713-9b42-efe1494a319f-kube-api-access-75mp8\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysctl-d\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102038    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-cni-bin\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysctl-conf\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.101987    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114e39d2-544d-4713-9b42-efe1494a319f-host-slash\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102042    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-systemd\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-ovn\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-cni-netd\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-lib-modules\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-systemd\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102201    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysctl-d\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102214    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-sysctl-conf\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102221    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-sys\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102251    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-ovn\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102273    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-cni-netd\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.103237 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102759    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-systemd\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.102245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-modprobe-d\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103353    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-host\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-slash\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103670    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-run-openvswitch\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103717    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103737    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-sys\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103801    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/850ce6be-0856-49f4-9b25-cf2a0396d1a5-host-slash\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103827    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-host\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103973    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/850ce6be-0856-49f4-9b25-cf2a0396d1a5-env-overrides\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.103979    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-lib-modules\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104659 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.104117    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-modprobe-d\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.104659 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.104506    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/850ce6be-0856-49f4-9b25-cf2a0396d1a5-ovn-node-metrics-cert\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.104934 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.104891    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cb737a7-e603-4199-86ec-e44ec1e67392-tmp\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.107898 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.107824    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cb737a7-e603-4199-86ec-e44ec1e67392-etc-tuned\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.115720 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.115697    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:02.115815 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.115728    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:02.115815 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.115742    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:02.115938 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.115822    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:33:02.615804008 +0000 UTC m=+3.177781375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:02.116782 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.116755    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27hc8\" (UniqueName: \"kubernetes.io/projected/cce8909d-08fb-45cb-a131-394cd09252a9-kube-api-access-27hc8\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:02.117476 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.117450    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whdm2\" (UniqueName: \"kubernetes.io/projected/9cb737a7-e603-4199-86ec-e44ec1e67392-kube-api-access-whdm2\") pod \"tuned-7p92d\" (UID: \"9cb737a7-e603-4199-86ec-e44ec1e67392\") " pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.120768 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.120744    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9njf\" (UniqueName: \"kubernetes.io/projected/ed690582-0a9e-4069-892c-78c07ea8066a-kube-api-access-l9njf\") pod \"node-ca-cpggg\" (UID: \"ed690582-0a9e-4069-892c-78c07ea8066a\") " pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.120998 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.120971    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75mp8\" (UniqueName: \"kubernetes.io/projected/114e39d2-544d-4713-9b42-efe1494a319f-kube-api-access-75mp8\") pod \"iptables-alerter-56rnv\" (UID: \"114e39d2-544d-4713-9b42-efe1494a319f\") " pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.121497 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.121462    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzhm7\" (UniqueName: \"kubernetes.io/projected/850ce6be-0856-49f4-9b25-cf2a0396d1a5-kube-api-access-hzhm7\") pod \"ovnkube-node-gx2vd\" (UID: \"850ce6be-0856-49f4-9b25-cf2a0396d1a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.180477 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.180448    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss"
Jul 08 16:33:02.188208 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.188185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:02.194709 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.194677    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec55c32f_1872_4d62_94e5_ff0c736bec34.slice/crio-5c7ef6fcc86548c90b1a90872e2ab9846c030e5a6e134839b2e46eb6575d2cb7 WatchSource:0}: Error finding container 5c7ef6fcc86548c90b1a90872e2ab9846c030e5a6e134839b2e46eb6575d2cb7: Status 404 returned error can't find the container with id 5c7ef6fcc86548c90b1a90872e2ab9846c030e5a6e134839b2e46eb6575d2cb7
Jul 08 16:33:02.197245 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.197219    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sr2lk"
Jul 08 16:33:02.202593 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.202534    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wzp4k"
Jul 08 16:33:02.205011 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.204982    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c511300_5a57_43ae_b976_92821d4b9b59.slice/crio-7f5ac679c6a4ef80d63a41f2c11693dc957e3d4972b4294936a715c41ccf5f86 WatchSource:0}: Error finding container 7f5ac679c6a4ef80d63a41f2c11693dc957e3d4972b4294936a715c41ccf5f86: Status 404 returned error can't find the container with id 7f5ac679c6a4ef80d63a41f2c11693dc957e3d4972b4294936a715c41ccf5f86
Jul 08 16:33:02.210501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.210481    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6jjf5"
Jul 08 16:33:02.210756 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.210734    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca18ed4_8fcd_451e_bdf8_6e7d4f66aa4f.slice/crio-f58bb130d4393bf9e91a51cb77270de471688fe95a2414be37eb098328a875d1 WatchSource:0}: Error finding container f58bb130d4393bf9e91a51cb77270de471688fe95a2414be37eb098328a875d1: Status 404 returned error can't find the container with id f58bb130d4393bf9e91a51cb77270de471688fe95a2414be37eb098328a875d1
Jul 08 16:33:02.214836 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.214806    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f281f6c_3a96_491b_bb22_f8ae225f25b1.slice/crio-4da172882b148148f19bdb4e7b458281b19d5192177b135359fd31435b874ef4 WatchSource:0}: Error finding container 4da172882b148148f19bdb4e7b458281b19d5192177b135359fd31435b874ef4: Status 404 returned error can't find the container with id 4da172882b148148f19bdb4e7b458281b19d5192177b135359fd31435b874ef4
Jul 08 16:33:02.223455 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.223432    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d57f4c7_60a7_47f1_8edb_3bcd4661c518.slice/crio-4e7cabb08d8d531b00289141e0b4b7429dba7149e6762279f37796cbacd7d494 WatchSource:0}: Error finding container 4e7cabb08d8d531b00289141e0b4b7429dba7149e6762279f37796cbacd7d494: Status 404 returned error can't find the container with id 4e7cabb08d8d531b00289141e0b4b7429dba7149e6762279f37796cbacd7d494
Jul 08 16:33:02.231315 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.231281    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-56rnv"
Jul 08 16:33:02.239223 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.239203    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7p92d"
Jul 08 16:33:02.240840 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.240814    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114e39d2_544d_4713_9b42_efe1494a319f.slice/crio-a871e5b573afb0d705aecdaf6be5633c93e1a9290a0609350175050b7fe56f1c WatchSource:0}: Error finding container a871e5b573afb0d705aecdaf6be5633c93e1a9290a0609350175050b7fe56f1c: Status 404 returned error can't find the container with id a871e5b573afb0d705aecdaf6be5633c93e1a9290a0609350175050b7fe56f1c
Jul 08 16:33:02.244670 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.244650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cpggg"
Jul 08 16:33:02.248037 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.248012    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb737a7_e603_4199_86ec_e44ec1e67392.slice/crio-8e93874a0ec413b73700cfb2985b36b249824e5f42a1a41f181435265beebba1 WatchSource:0}: Error finding container 8e93874a0ec413b73700cfb2985b36b249824e5f42a1a41f181435265beebba1: Status 404 returned error can't find the container with id 8e93874a0ec413b73700cfb2985b36b249824e5f42a1a41f181435265beebba1
Jul 08 16:33:02.249661 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.249642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:02.254899 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.254850    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded690582_0a9e_4069_892c_78c07ea8066a.slice/crio-574690c22c5d0f018d6215e3242472c0b3ee281ec486e25f90ae3146931f9a18 WatchSource:0}: Error finding container 574690c22c5d0f018d6215e3242472c0b3ee281ec486e25f90ae3146931f9a18: Status 404 returned error can't find the container with id 574690c22c5d0f018d6215e3242472c0b3ee281ec486e25f90ae3146931f9a18
Jul 08 16:33:02.261860 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:02.261823    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850ce6be_0856_49f4_9b25_cf2a0396d1a5.slice/crio-b26806a927e0a4f86532287ce593c74f0c12077f16250c6d603008b7a3719f4c WatchSource:0}: Error finding container b26806a927e0a4f86532287ce593c74f0c12077f16250c6d603008b7a3719f4c: Status 404 returned error can't find the container with id b26806a927e0a4f86532287ce593c74f0c12077f16250c6d603008b7a3719f4c
Jul 08 16:33:02.507246 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.507152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:02.507411 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.507384    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:02.507478 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.507455    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:33:03.507434389 +0000 UTC m=+4.069411770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:02.608616 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.608577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:02.608798 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.608710    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:02.608876 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.608799    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:03.608775197 +0000 UTC m=+4.170752561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:02.709420 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.709377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:02.709597 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.709568    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:02.709678 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.709599    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:02.709678 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.709613    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:02.709881 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:02.709682    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:33:03.709662522 +0000 UTC m=+4.271639891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:02.930005 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.929897    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:28:01 +0000 UTC" deadline="2027-12-15 10:51:50.36139694 +0000 UTC"
Jul 08 16:33:02.930005 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:02.929958    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12594h18m47.431442941s"
Jul 08 16:33:03.023816 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.023779    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:03.024016 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.023936    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:03.034584 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.034498    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6jjf5" event={"ID":"1d57f4c7-60a7-47f1-8edb-3bcd4661c518","Type":"ContainerStarted","Data":"4e7cabb08d8d531b00289141e0b4b7429dba7149e6762279f37796cbacd7d494"}
Jul 08 16:33:03.035900 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.035865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerStarted","Data":"f58bb130d4393bf9e91a51cb77270de471688fe95a2414be37eb098328a875d1"}
Jul 08 16:33:03.037116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.037085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss" event={"ID":"ec55c32f-1872-4d62-94e5-ff0c736bec34","Type":"ContainerStarted","Data":"5c7ef6fcc86548c90b1a90872e2ab9846c030e5a6e134839b2e46eb6575d2cb7"}
Jul 08 16:33:03.038414 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.038390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"b26806a927e0a4f86532287ce593c74f0c12077f16250c6d603008b7a3719f4c"}
Jul 08 16:33:03.039714 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.039680    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cpggg" event={"ID":"ed690582-0a9e-4069-892c-78c07ea8066a","Type":"ContainerStarted","Data":"574690c22c5d0f018d6215e3242472c0b3ee281ec486e25f90ae3146931f9a18"}
Jul 08 16:33:03.040887 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.040865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-56rnv" event={"ID":"114e39d2-544d-4713-9b42-efe1494a319f","Type":"ContainerStarted","Data":"a871e5b573afb0d705aecdaf6be5633c93e1a9290a0609350175050b7fe56f1c"}
Jul 08 16:33:03.042020 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.041983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wzp4k" event={"ID":"4f281f6c-3a96-491b-bb22-f8ae225f25b1","Type":"ContainerStarted","Data":"4da172882b148148f19bdb4e7b458281b19d5192177b135359fd31435b874ef4"}
Jul 08 16:33:03.043175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.043148    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-89vm6" event={"ID":"1c511300-5a57-43ae-b976-92821d4b9b59","Type":"ContainerStarted","Data":"7f5ac679c6a4ef80d63a41f2c11693dc957e3d4972b4294936a715c41ccf5f86"}
Jul 08 16:33:03.044378 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.044352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7p92d" event={"ID":"9cb737a7-e603-4199-86ec-e44ec1e67392","Type":"ContainerStarted","Data":"8e93874a0ec413b73700cfb2985b36b249824e5f42a1a41f181435265beebba1"}
Jul 08 16:33:03.515621 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.515003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:03.515621 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.515164    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:03.515621 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.515247    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:33:05.515216658 +0000 UTC m=+6.077194025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:03.616030 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.615982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:03.616240 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.616222    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:03.616312 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.616304    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:05.616283245 +0000 UTC m=+6.178260628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:03.716979 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:03.716934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:03.717218 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.717200    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:03.717295 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.717224    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:03.717295 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.717238    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:03.717415 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:03.717296    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:33:05.71727815 +0000 UTC m=+6.279255524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:04.026450 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:04.026415    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:04.026612 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:04.026552    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:04.027051 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:04.027019    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:04.027160 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:04.027145    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:04.053710 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:04.053231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal" event={"ID":"43f0a89a04d165574d4c36c955b6bc1e","Type":"ContainerStarted","Data":"249b0535e8fc4b3669f21c6bc7e514df842236417b3d1c34a6718a47e3aced7b"}
Jul 08 16:33:05.023489 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.023424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:05.023663 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.023557    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:05.074090 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.074025    2562 generic.go:356] "Generic (PLEG): container finished" podID="19047cfbf21b92e7ac69508e6d358f35" containerID="459d8c6df7049944568fdbb177c3ee8bc818deb2e6f16b3f82d94dbea3c5f44b" exitCode=0
Jul 08 16:33:05.074860 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.074777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal" event={"ID":"19047cfbf21b92e7ac69508e6d358f35","Type":"ContainerDied","Data":"459d8c6df7049944568fdbb177c3ee8bc818deb2e6f16b3f82d94dbea3c5f44b"}
Jul 08 16:33:05.096172 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.096117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-182.ec2.internal" podStartSLOduration=4.096097192 podStartE2EDuration="4.096097192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:04.072159187 +0000 UTC m=+4.634136576" watchObservedRunningTime="2026-07-08 16:33:05.096097192 +0000 UTC m=+5.658074581"
Jul 08 16:33:05.534467 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.534071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:05.534467 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.534247    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:05.534467 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.534315    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:33:09.534296529 +0000 UTC m=+10.096273897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:05.635274 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.635233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:05.635467 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.635414    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:05.635531 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.635488    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:09.635466575 +0000 UTC m=+10.197443942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:05.736025 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:05.735987    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:05.736230 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.736191    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:05.736230 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.736214    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:05.736230 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.736226    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:05.736391 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:05.736289    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:33:09.736269517 +0000 UTC m=+10.298246882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:06.026593 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:06.026557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:06.026795 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:06.026702    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:06.027155 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:06.027128    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:06.027321 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:06.027293    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:06.087970 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:06.087933    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal" event={"ID":"19047cfbf21b92e7ac69508e6d358f35","Type":"ContainerStarted","Data":"8074e6aebf99e56770759badeb5f896c72cf791279607ea784ee8af490ba457c"}
Jul 08 16:33:07.023099 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:07.023051    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:07.023312 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:07.023165    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:08.024062 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:08.024028    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:08.024542 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:08.024167    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:08.024542 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:08.024229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:08.024542 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:08.024380    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:09.023153 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:09.023115    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:09.023350 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.023259    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:09.571227 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:09.571167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:09.571731 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.571344    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:09.571731 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.571425    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:33:17.571403915 +0000 UTC m=+18.133381279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:09.672915 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:09.672310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:09.672915 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.672454    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:09.672915 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.672530    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:17.672509017 +0000 UTC m=+18.234486382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:09.773519 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:09.773474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:09.773712 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.773686    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:09.773712 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.773711    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:09.773840 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.773725    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:09.773840 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:09.773815    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:33:17.773796783 +0000 UTC m=+18.335774149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:10.024077 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:10.024041    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:10.024238 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:10.024174    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:10.024238 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:10.024215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:10.024341 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:10.024317    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:11.024323 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:11.024055    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:11.024323 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:11.024195    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:12.023730 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:12.023697    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:12.023946 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:12.023849    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:12.023946 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:12.023882    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:12.024052 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:12.024028    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:13.023304 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:13.023272    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:13.023748 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:13.023396    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:14.024023 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:14.023992    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:14.024465 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:14.024000    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:14.024465 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:14.024116    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:14.024465 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:14.024205    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:15.023185 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:15.023151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:15.023375 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:15.023283    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:16.023219 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:16.023178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:16.023738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:16.023189    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:16.023738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:16.023333    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:16.023738 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:16.023398    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:17.022948 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:17.022919    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:17.023138 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.023022    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:17.635259 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:17.635233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:17.635583 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.635374    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:17.635583 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.635432    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:33:33.635412804 +0000 UTC m=+34.197390166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:17.735746 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:17.735715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:17.735909 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.735839    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:17.735909 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.735888    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:33.735873845 +0000 UTC m=+34.297851207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:17.836397 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:17.836362    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:17.836554 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.836527    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:17.836554 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.836551    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:17.836691 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.836562    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:17.836691 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:17.836612    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:33:33.83659904 +0000 UTC m=+34.398576401 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:18.023508 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:18.023474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:18.023662 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:18.023601    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:18.023721 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:18.023691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:18.023860 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:18.023812    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:19.023869 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:19.023838    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:19.024273 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:19.023968    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:20.026170 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:20.024785    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:20.026170 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:20.024938    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:20.026170 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:20.025378    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:20.026170 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:20.025509    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:21.024017 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:21.023980    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:21.024169 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:21.024107    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:22.026030 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:22.026005    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:22.026418 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:22.026131    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:22.026482 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:22.026443    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:22.026560 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:22.026540    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:22.122126 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:22.122007    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7p92d" event={"ID":"9cb737a7-e603-4199-86ec-e44ec1e67392","Type":"ContainerStarted","Data":"42f3b0a49f48fcf5b9a76f424327952f7499aea53a33bb090dee91468841ea78"}
Jul 08 16:33:22.146434 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:22.146265    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-182.ec2.internal" podStartSLOduration=21.146249158 podStartE2EDuration="21.146249158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:06.108818183 +0000 UTC m=+6.670795592" watchObservedRunningTime="2026-07-08 16:33:22.146249158 +0000 UTC m=+22.708226557"
Jul 08 16:33:23.023277 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.023104    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:23.023411 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:23.023349    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:23.126991 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.126948    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"e938d98120564834a7376e666ac96da9c25a34509ffc3a72f217f973f2c52866"}
Jul 08 16:33:23.127756 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.127020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"19ec1b0a509eb5d6decb06c654d5a36f236f6a8bdf2d9e2202130b3eca7f4937"}
Jul 08 16:33:23.127756 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.127063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"9e46f182a67e74632154d04886f978fff6631fd9d451a452a3ea5d1a21d45027"}
Jul 08 16:33:23.127756 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.127096    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"cffa3b15a748e33ae4ef709cad3d68fab7d6bcaab60ba6e72c4d502ff4ce9198"}
Jul 08 16:33:23.127756 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.127123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"0823c6c4e2e015c94e2935919ebd4a1bbd9e8dcfa8b38844fc15ee14d8bba944"}
Jul 08 16:33:23.127756 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.127169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"d35babd6e13be7c155b256baa24bd83d2dd335fcdc4b47b67a152ffc06eae9e8"}
Jul 08 16:33:23.128370 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.128340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cpggg" event={"ID":"ed690582-0a9e-4069-892c-78c07ea8066a","Type":"ContainerStarted","Data":"88061764d09f40935141f3abb45508665d3fd50d5b8f2afbd62f0febbf4408d3"}
Jul 08 16:33:23.129888 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.129863    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wzp4k" event={"ID":"4f281f6c-3a96-491b-bb22-f8ae225f25b1","Type":"ContainerStarted","Data":"a31f3d19b9d8db50119e853d25e0f323986a34ea269b8a4bdabf824e1a4f0cdb"}
Jul 08 16:33:23.131270 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.131247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-89vm6" event={"ID":"1c511300-5a57-43ae-b976-92821d4b9b59","Type":"ContainerStarted","Data":"18d028b89dbe66e4efa18b1eb128441b2f646af9193b592adb9cc4aa07062121"}
Jul 08 16:33:23.132543 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.132525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6jjf5" event={"ID":"1d57f4c7-60a7-47f1-8edb-3bcd4661c518","Type":"ContainerStarted","Data":"e35db6299fb55ba0a2810e0355fb5a926f989f35ac59fbe4f8e756453c7ce094"}
Jul 08 16:33:23.133912 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.133892    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f" containerID="776a41ef2389ba879b38d1a99c7632feb4fa8ca86369279705774deca5c0c059" exitCode=0
Jul 08 16:33:23.133991 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.133954    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerDied","Data":"776a41ef2389ba879b38d1a99c7632feb4fa8ca86369279705774deca5c0c059"}
Jul 08 16:33:23.135343 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.135323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss" event={"ID":"ec55c32f-1872-4d62-94e5-ff0c736bec34","Type":"ContainerStarted","Data":"626c755b770f89ce7541ed3d5196ea745fb2c1354d4a12cfa74df671de77a113"}
Jul 08 16:33:23.150190 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.150154    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7p92d" podStartSLOduration=2.418997895 podStartE2EDuration="22.150142281s" totalImagesPullingTime="19.731144386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:22.145946927 +0000 UTC m=+22.707924312" watchObservedRunningTime="2026-07-08 16:33:23.150142281 +0000 UTC m=+23.712119665"
Jul 08 16:33:23.150272 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.150240    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cpggg" podStartSLOduration=2.428848141 podStartE2EDuration="22.150234725s" totalImagesPullingTime="19.721386584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:23.149548321 +0000 UTC m=+23.711525708" watchObservedRunningTime="2026-07-08 16:33:23.150234725 +0000 UTC m=+23.712212108"
Jul 08 16:33:23.185947 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.185908    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-89vm6" podStartSLOduration=3.414755657 podStartE2EDuration="23.185894423s" totalImagesPullingTime="19.771138766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:23.164431128 +0000 UTC m=+23.726408512" watchObservedRunningTime="2026-07-08 16:33:23.185894423 +0000 UTC m=+23.747871809"
Jul 08 16:33:23.200574 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.200534    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wzp4k" podStartSLOduration=6.839288538 podStartE2EDuration="22.200519854s" totalImagesPullingTime="15.361231316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:23.20049463 +0000 UTC m=+23.762472015" watchObservedRunningTime="2026-07-08 16:33:23.200519854 +0000 UTC m=+23.762497237"
Jul 08 16:33:23.220755 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.220716    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6jjf5" podStartSLOduration=2.408473327 podStartE2EDuration="22.220706716s" totalImagesPullingTime="19.812233389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:23.220457885 +0000 UTC m=+23.782435269" watchObservedRunningTime="2026-07-08 16:33:23.220706716 +0000 UTC m=+23.782684101"
Jul 08 16:33:23.799555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:23.799527    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 16:33:24.023569 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.023491    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:24.023779 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:24.023624    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:24.023779 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.023646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:24.023915 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:24.023781    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:24.028032 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.027942    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T16:33:23.79954899Z","UUID":"79d08fe6-05a8-4601-a5de-340badc9f9e0","Handler":null,"Name":"","Endpoint":""}
Jul 08 16:33:24.029502 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.029485    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 16:33:24.029590 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.029522    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 16:33:24.139486 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.139451    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss" event={"ID":"ec55c32f-1872-4d62-94e5-ff0c736bec34","Type":"ContainerStarted","Data":"08817fa1ac25ed9efc3edaba2465a4dde75fd5ef178a58bf7d2273c85ce27e1b"}
Jul 08 16:33:24.140802 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.140772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-56rnv" event={"ID":"114e39d2-544d-4713-9b42-efe1494a319f","Type":"ContainerStarted","Data":"77a384a8d614f2db7b06f1c1c4f067dca5af58ef6106002fa086cfed31ce9ec0"}
Jul 08 16:33:24.155572 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:24.155528    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-56rnv" podStartSLOduration=7.821477983 podStartE2EDuration="23.155513729s" totalImagesPullingTime="15.334035746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:24.15491563 +0000 UTC m=+24.716893015" watchObservedRunningTime="2026-07-08 16:33:24.155513729 +0000 UTC m=+24.717491112"
Jul 08 16:33:25.023121 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:25.023095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:25.023345 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:25.023180    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:25.146894 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:25.146857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"ab20c159312993fb75be4b44023a8218330dbbda75a88a2235cce58117805a82"}
Jul 08 16:33:26.023360 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:26.023150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:26.023564 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:26.023229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:26.023564 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:26.023469    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:26.023691 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:26.023561    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:26.151297 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:26.151262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss" event={"ID":"ec55c32f-1872-4d62-94e5-ff0c736bec34","Type":"ContainerStarted","Data":"b6b2ea5f1fa0778bd8a471babb8acd2c7d7161c664360c792f91dea7b4117909"}
Jul 08 16:33:26.171182 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:26.171127    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swsss" podStartSLOduration=3.514382408 podStartE2EDuration="26.171106968s" totalImagesPullingTime="22.65672456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:26.17080702 +0000 UTC m=+26.732784404" watchObservedRunningTime="2026-07-08 16:33:26.171106968 +0000 UTC m=+26.733084353"
Jul 08 16:33:27.023769 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:27.023735    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:27.023980 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:27.023850    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:27.394734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:27.394704    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:27.395465 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:27.395450    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:28.023362 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:28.023315    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:28.023540 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:28.023322    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:28.023540 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:28.023452    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:28.023654 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:28.023560    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:28.159949 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:28.159896    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" event={"ID":"850ce6be-0856-49f4-9b25-cf2a0396d1a5","Type":"ContainerStarted","Data":"6e10cabb364541fc0608e53bae6043e5820a156f16f582042e8608b417215cbc"}
Jul 08 16:33:28.160255 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:28.160231    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:28.160701 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:28.160681    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-89vm6"
Jul 08 16:33:28.190547 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:28.190498    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd" podStartSLOduration=7.414285015 podStartE2EDuration="27.190479251s" totalImagesPullingTime="19.776194236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:28.188659602 +0000 UTC m=+28.750636986" watchObservedRunningTime="2026-07-08 16:33:28.190479251 +0000 UTC m=+28.752456637"
Jul 08 16:33:29.023712 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.023676    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:29.024169 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:29.023815    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:29.163052 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.163023    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:29.163225 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.163107    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:29.163225 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.163151    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:29.181047 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.181017    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:29.181753 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.181727    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:33:29.532089 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.532066    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mc44p"]
Jul 08 16:33:29.532204 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.532191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:29.532322 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:29.532303    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:29.536559 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.536536    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-br9fz"]
Jul 08 16:33:29.536662 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.536650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:29.536769 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:29.536753    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:29.537689 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.537666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-64tq5"]
Jul 08 16:33:29.537764 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:29.537740    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:29.537890 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:29.537814    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:30.166357 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:30.166323    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f" containerID="2bef5384f5ae1fa95971a0b3d89670e0e4da94a548033de4ee9fe74096fea261" exitCode=0
Jul 08 16:33:30.166914 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:30.166406    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerDied","Data":"2bef5384f5ae1fa95971a0b3d89670e0e4da94a548033de4ee9fe74096fea261"}
Jul 08 16:33:31.023874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:31.023846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:31.023874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:31.023865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:31.023874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:31.023891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:31.024091 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:31.023966    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:31.024142 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:31.024085    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:31.024179 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:31.024149    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:32.171838 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:32.171569    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f" containerID="303d30f2e8cf9c7930a9398e647b07d9b3351ee53e4f4e48c1626a24368abe4e" exitCode=0
Jul 08 16:33:32.172240 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:32.171651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerDied","Data":"303d30f2e8cf9c7930a9398e647b07d9b3351ee53e4f4e48c1626a24368abe4e"}
Jul 08 16:33:33.023752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:33.023726    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:33.023939 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:33.023816    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:33.023939 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.023838    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:33.023939 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:33.023836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:33.023939 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.023916    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:33.024069 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.023981    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:33.657235 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:33.657206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:33.657762 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.657303    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:33.657762 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.657352    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret podName:e967c6f7-5197-4648-8ce2-cffcd788026d nodeName:}" failed. No retries permitted until 2026-07-08 16:34:05.657338347 +0000 UTC m=+66.219315710 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret") pod "global-pull-secret-syncer-mc44p" (UID: "e967c6f7-5197-4648-8ce2-cffcd788026d") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:33:33.757721 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:33.757690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:33.757886 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.757798    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:33.757886 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.757869    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs podName:cce8909d-08fb-45cb-a131-394cd09252a9 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:05.7578515 +0000 UTC m=+66.319828880 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs") pod "network-metrics-daemon-br9fz" (UID: "cce8909d-08fb-45cb-a131-394cd09252a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:33:33.858137 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:33.858100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:33.858278 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.858253    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:33:33.858278 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.858273    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:33:33.858337 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.858283    2562 projected.go:196] Error preparing data for projected volume kube-api-access-lgqcb for pod openshift-network-diagnostics/network-check-target-64tq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:33.858337 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:33.858328    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb podName:f75e0ea9-22bb-495a-99bc-117ccfccde9b nodeName:}" failed. No retries permitted until 2026-07-08 16:34:05.858315836 +0000 UTC m=+66.420293201 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgqcb" (UniqueName: "kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb") pod "network-check-target-64tq5" (UID: "f75e0ea9-22bb-495a-99bc-117ccfccde9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:33:34.178503 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:34.178427    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f" containerID="e59b27ae30ffb2c6ee66f1a6041aaca02cd5f9ff0bf5a033fd19d2a7720269a9" exitCode=0
Jul 08 16:33:34.178503 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:34.178479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerDied","Data":"e59b27ae30ffb2c6ee66f1a6041aaca02cd5f9ff0bf5a033fd19d2a7720269a9"}
Jul 08 16:33:35.023217 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.023183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:35.023665 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.023231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:35.023665 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.023398    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64tq5" podUID="f75e0ea9-22bb-495a-99bc-117ccfccde9b"
Jul 08 16:33:35.023665 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.023501    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc44p" podUID="e967c6f7-5197-4648-8ce2-cffcd788026d"
Jul 08 16:33:35.023665 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.023263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:35.023665 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.023617    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-br9fz" podUID="cce8909d-08fb-45cb-a131-394cd09252a9"
Jul 08 16:33:35.653127 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.653096    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-182.ec2.internal" event="NodeReady"
Jul 08 16:33:35.653345 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.653257    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 16:33:35.688942 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.688908    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6db7d84484-znjz9"]
Jul 08 16:33:35.711087 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.711060    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vctc7"]
Jul 08 16:33:35.711249 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.711191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.713945 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.713918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 16:33:35.714095 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.714031    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dvnjd\""
Jul 08 16:33:35.714195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.714156    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 16:33:35.714675 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.714659    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 16:33:35.719554 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.719030    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-56gsq"]
Jul 08 16:33:35.719554 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.719157    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.720961 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.720520    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 16:33:35.721665 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.721646    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 16:33:35.721927 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.721910    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 16:33:35.722017 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.721928    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4gk8\""
Jul 08 16:33:35.726440 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.726420    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6db7d84484-znjz9"]
Jul 08 16:33:35.726534 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.726455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vctc7"]
Jul 08 16:33:35.726598 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.726546    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:35.729279 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.729071    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 16:33:35.729279 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.729149    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 16:33:35.729279 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.729182    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 16:33:35.729525 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.729323    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-twqsl\""
Jul 08 16:33:35.732447 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.732430    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-56gsq"]
Jul 08 16:33:35.874369 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-trusted-ca\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/539de09f-b3b6-4ed0-8e63-284856086968-tmp-dir\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.874552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk8gv\" (UniqueName: \"kubernetes.io/projected/539de09f-b3b6-4ed0-8e63-284856086968-kube-api-access-qk8gv\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.874552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/894c4f16-253b-4e22-8fc7-1fe5aa158afc-ca-trust-extracted\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-installation-pull-secrets\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-bound-sa-token\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/539de09f-b3b6-4ed0-8e63-284856086968-config-volume\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-image-registry-private-configuration\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr9n4\" (UniqueName: \"kubernetes.io/projected/7130273a-98c0-46d9-b77e-ed40a3bb73be-kube-api-access-nr9n4\") pod \"ingress-canary-56gsq\" (UID: \"7130273a-98c0-46d9-b77e-ed40a3bb73be\") " pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-certificates\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs5fd\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-kube-api-access-vs5fd\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7130273a-98c0-46d9-b77e-ed40a3bb73be-cert\") pod \"ingress-canary-56gsq\" (UID: \"7130273a-98c0-46d9-b77e-ed40a3bb73be\") " pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:35.874901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.874846    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.975298 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975262    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.975487 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-trusted-ca\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975487 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/539de09f-b3b6-4ed0-8e63-284856086968-tmp-dir\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.975487 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk8gv\" (UniqueName: \"kubernetes.io/projected/539de09f-b3b6-4ed0-8e63-284856086968-kube-api-access-qk8gv\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.975487 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.975427    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 16:33:35.975487 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/894c4f16-253b-4e22-8fc7-1fe5aa158afc-ca-trust-extracted\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-installation-pull-secrets\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-bound-sa-token\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/539de09f-b3b6-4ed0-8e63-284856086968-config-volume\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-image-registry-private-configuration\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975610    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr9n4\" (UniqueName: \"kubernetes.io/projected/7130273a-98c0-46d9-b77e-ed40a3bb73be-kube-api-access-nr9n4\") pod \"ingress-canary-56gsq\" (UID: \"7130273a-98c0-46d9-b77e-ed40a3bb73be\") " pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-certificates\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs5fd\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-kube-api-access-vs5fd\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.975754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7130273a-98c0-46d9-b77e-ed40a3bb73be-cert\") pod \"ingress-canary-56gsq\" (UID: \"7130273a-98c0-46d9-b77e-ed40a3bb73be\") " pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:35.976210 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.975769    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/539de09f-b3b6-4ed0-8e63-284856086968-tmp-dir\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.976210 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.975802    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls podName:539de09f-b3b6-4ed0-8e63-284856086968 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:36.475774099 +0000 UTC m=+37.037751468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls") pod "dns-default-vctc7" (UID: "539de09f-b3b6-4ed0-8e63-284856086968") : secret "dns-default-metrics-tls" not found
Jul 08 16:33:35.976210 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.975860    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 16:33:35.976210 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.975874    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db7d84484-znjz9: secret "image-registry-tls" not found
Jul 08 16:33:35.976210 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:35.975931    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls podName:894c4f16-253b-4e22-8fc7-1fe5aa158afc nodeName:}" failed. No retries permitted until 2026-07-08 16:33:36.475915671 +0000 UTC m=+37.037893057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls") pod "image-registry-6db7d84484-znjz9" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc") : secret "image-registry-tls" not found
Jul 08 16:33:35.976210 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.976043    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/894c4f16-253b-4e22-8fc7-1fe5aa158afc-ca-trust-extracted\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.976582 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.976263    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-certificates\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.976582 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.976467    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-trusted-ca\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.976582 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.976493    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/539de09f-b3b6-4ed0-8e63-284856086968-config-volume\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.980402 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.980377    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-installation-pull-secrets\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.980516 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.980455    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7130273a-98c0-46d9-b77e-ed40a3bb73be-cert\") pod \"ingress-canary-56gsq\" (UID: \"7130273a-98c0-46d9-b77e-ed40a3bb73be\") " pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:35.980516 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.980458    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-image-registry-private-configuration\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.984472 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.984447    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs5fd\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-kube-api-access-vs5fd\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.984678 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.984657    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk8gv\" (UniqueName: \"kubernetes.io/projected/539de09f-b3b6-4ed0-8e63-284856086968-kube-api-access-qk8gv\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:35.985354 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.985328    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-bound-sa-token\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:35.985614 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:35.985599    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr9n4\" (UniqueName: \"kubernetes.io/projected/7130273a-98c0-46d9-b77e-ed40a3bb73be-kube-api-access-nr9n4\") pod \"ingress-canary-56gsq\" (UID: \"7130273a-98c0-46d9-b77e-ed40a3bb73be\") " pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:36.039885 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.039855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-56gsq"
Jul 08 16:33:36.244301 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.244136    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-56gsq"]
Jul 08 16:33:36.247126 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:36.247095    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7130273a_98c0_46d9_b77e_ed40a3bb73be.slice/crio-3d31bf67e6be39717910d375c23e97350d9105111850d23afab771bde5112dfc WatchSource:0}: Error finding container 3d31bf67e6be39717910d375c23e97350d9105111850d23afab771bde5112dfc: Status 404 returned error can't find the container with id 3d31bf67e6be39717910d375c23e97350d9105111850d23afab771bde5112dfc
Jul 08 16:33:36.480976 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.480850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:36.480976 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.480938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:36.481200 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.481006    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 16:33:36.481200 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.481028    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db7d84484-znjz9: secret "image-registry-tls" not found
Jul 08 16:33:36.481200 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.481066    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 16:33:36.481200 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.481083    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls podName:894c4f16-253b-4e22-8fc7-1fe5aa158afc nodeName:}" failed. No retries permitted until 2026-07-08 16:33:37.481068007 +0000 UTC m=+38.043045369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls") pod "image-registry-6db7d84484-znjz9" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc") : secret "image-registry-tls" not found
Jul 08 16:33:36.481200 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.481191    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls podName:539de09f-b3b6-4ed0-8e63-284856086968 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:37.481175637 +0000 UTC m=+38.043153005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls") pod "dns-default-vctc7" (UID: "539de09f-b3b6-4ed0-8e63-284856086968") : secret "dns-default-metrics-tls" not found
Jul 08 16:33:36.640399 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.640355    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"]
Jul 08 16:33:36.673520 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.673497    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"]
Jul 08 16:33:36.673670 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.673584    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.676584 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.676559    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 16:33:36.677639 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.677620    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 08 16:33:36.677745 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.677676    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-77lg6\""
Jul 08 16:33:36.679219 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.679186    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 16:33:36.679325 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.679255    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 08 16:33:36.783232 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.783158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b45abeec-9848-44ca-b486-42f75a62b891-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.783232 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.783214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.783446 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.783369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6xwd\" (UniqueName: \"kubernetes.io/projected/b45abeec-9848-44ca-b486-42f75a62b891-kube-api-access-r6xwd\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.800148 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.800119    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"]
Jul 08 16:33:36.812206 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.812183    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"]
Jul 08 16:33:36.812339 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.812307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"
Jul 08 16:33:36.814865 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.814812    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:36.815098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.815080    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:36.815417 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.815401    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4p97r\""
Jul 08 16:33:36.832192 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.832172    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xh8sq"]
Jul 08 16:33:36.832306 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.832291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:36.835137 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.835116    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-flbkm\""
Jul 08 16:33:36.835243 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.835117    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 08 16:33:36.835243 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.835160    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:36.835243 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.835184    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:36.835243 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.835209    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 08 16:33:36.854651 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.854629    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"]
Jul 08 16:33:36.854785 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.854664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"]
Jul 08 16:33:36.854785 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.854682    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xh8sq"]
Jul 08 16:33:36.854785 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.854775    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:36.857563 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.857461    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 08 16:33:36.857649 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.857615    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 08 16:33:36.857716 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.857687    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 16:33:36.858136 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.858115    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 16:33:36.858225 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.858141    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4vqt4\""
Jul 08 16:33:36.862730 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.862710    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 08 16:33:36.884539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.884412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.884539 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.884520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xwd\" (UniqueName: \"kubernetes.io/projected/b45abeec-9848-44ca-b486-42f75a62b891-kube-api-access-r6xwd\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.884539 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.884527    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:36.884757 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:36.884599    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls podName:b45abeec-9848-44ca-b486-42f75a62b891 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:37.384579173 +0000 UTC m=+37.946556547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xgl2r" (UID: "b45abeec-9848-44ca-b486-42f75a62b891") : secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:36.884757 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.884622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b45abeec-9848-44ca-b486-42f75a62b891-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.885482 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.885461    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b45abeec-9848-44ca-b486-42f75a62b891-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.896191 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.896168    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6xwd\" (UniqueName: \"kubernetes.io/projected/b45abeec-9848-44ca-b486-42f75a62b891-kube-api-access-r6xwd\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:36.904717 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.904692    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"]
Jul 08 16:33:36.923060 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.923034    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"]
Jul 08 16:33:36.923174 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.923152    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:36.925714 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.925690    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:36.926063 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.925925    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:36.926063 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.925953    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 08 16:33:36.926063 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.926009    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 08 16:33:36.926265 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.926216    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c6hxr\""
Jul 08 16:33:36.985244 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/113bd2ab-b61a-41f1-a106-0336460bfae4-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:36.985376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db950e2c-5b14-484d-941d-879cff4b3343-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:36.985376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/113bd2ab-b61a-41f1-a106-0336460bfae4-config\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:36.985376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f26ps\" (UniqueName: \"kubernetes.io/projected/db950e2c-5b14-484d-941d-879cff4b3343-kube-api-access-f26ps\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:36.985376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985350    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rnxn\" (UniqueName: \"kubernetes.io/projected/5cc57719-0da2-430d-8f48-7ef9d9d774fd-kube-api-access-2rnxn\") pod \"volume-data-source-validator-5d456fdbcd-29gdq\" (UID: \"5cc57719-0da2-430d-8f48-7ef9d9d774fd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"
Jul 08 16:33:36.985499 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db950e2c-5b14-484d-941d-879cff4b3343-serving-cert\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:36.985499 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db950e2c-5b14-484d-941d-879cff4b3343-tmp\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:36.985561 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db950e2c-5b14-484d-941d-879cff4b3343-snapshots\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:36.985561 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985533    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh29d\" (UniqueName: \"kubernetes.io/projected/113bd2ab-b61a-41f1-a106-0336460bfae4-kube-api-access-wh29d\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:36.985561 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:36.985556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db950e2c-5b14-484d-941d-879cff4b3343-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.002185 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.002159    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"]
Jul 08 16:33:37.023952 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.022995    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tnz7d"]
Jul 08 16:33:37.023952 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.023682    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.026038 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.026015    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:37.026170 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.026018    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:37.026381 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.026356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 08 16:33:37.026381 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.026376    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-z8tdd\""
Jul 08 16:33:37.049284 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.049226    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7df5b6f649-fxv58"]
Jul 08 16:33:37.049634 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.049353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.049634 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.049363    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:33:37.049634 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.049384    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:33:37.049634 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.049354    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:33:37.052177 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.051997    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 16:33:37.052177 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052008    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s7kjq\""
Jul 08 16:33:37.052177 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052029    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 16:33:37.052177 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052008    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 08 16:33:37.052398 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052352    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:37.052481 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052460    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4cx7\""
Jul 08 16:33:37.052481 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052479    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-29rcb\""
Jul 08 16:33:37.052652 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052499    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 16:33:37.052652 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052624    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 08 16:33:37.052752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052730    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 08 16:33:37.052924 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.052908    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 16:33:37.057078 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.057059    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 08 16:33:37.065203 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.065183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"]
Jul 08 16:33:37.065295 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.065223    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tnz7d"]
Jul 08 16:33:37.065295 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.065241    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7df5b6f649-fxv58"]
Jul 08 16:33:37.065499 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.065351    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.067633 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.067614    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 16:33:37.067874 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.067858    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 08 16:33:37.067950 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.067918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-27cr8\""
Jul 08 16:33:37.067950 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.067935    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 08 16:33:37.068077 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.068059    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 08 16:33:37.068175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.068158    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 08 16:33:37.068291 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.068278    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 16:33:37.086789 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086600    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db950e2c-5b14-484d-941d-879cff4b3343-snapshots\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.086789 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh29d\" (UniqueName: \"kubernetes.io/projected/113bd2ab-b61a-41f1-a106-0336460bfae4-kube-api-access-wh29d\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.086789 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db950e2c-5b14-484d-941d-879cff4b3343-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.086789 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12d88f79-f370-45df-9935-944f43c1c1d7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.086789 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/113bd2ab-b61a-41f1-a106-0336460bfae4-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db950e2c-5b14-484d-941d-879cff4b3343-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/113bd2ab-b61a-41f1-a106-0336460bfae4-config\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f26ps\" (UniqueName: \"kubernetes.io/projected/db950e2c-5b14-484d-941d-879cff4b3343-kube-api-access-f26ps\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086947    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8svd\" (UniqueName: \"kubernetes.io/projected/12d88f79-f370-45df-9935-944f43c1c1d7-kube-api-access-h8svd\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.086993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rnxn\" (UniqueName: \"kubernetes.io/projected/5cc57719-0da2-430d-8f48-7ef9d9d774fd-kube-api-access-2rnxn\") pod \"volume-data-source-validator-5d456fdbcd-29gdq\" (UID: \"5cc57719-0da2-430d-8f48-7ef9d9d774fd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.087019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db950e2c-5b14-484d-941d-879cff4b3343-serving-cert\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.087069    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12d88f79-f370-45df-9935-944f43c1c1d7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.087115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.087111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db950e2c-5b14-484d-941d-879cff4b3343-tmp\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.087916 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.087528    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db950e2c-5b14-484d-941d-879cff4b3343-tmp\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.087999 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.087965    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db950e2c-5b14-484d-941d-879cff4b3343-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.088457 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.088079    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/113bd2ab-b61a-41f1-a106-0336460bfae4-config\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.088457 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.088337    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db950e2c-5b14-484d-941d-879cff4b3343-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.088611 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.088524    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db950e2c-5b14-484d-941d-879cff4b3343-snapshots\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.089752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.089730    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/113bd2ab-b61a-41f1-a106-0336460bfae4-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.091028 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.091006    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db950e2c-5b14-484d-941d-879cff4b3343-serving-cert\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.097290 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.097248    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rnxn\" (UniqueName: \"kubernetes.io/projected/5cc57719-0da2-430d-8f48-7ef9d9d774fd-kube-api-access-2rnxn\") pod \"volume-data-source-validator-5d456fdbcd-29gdq\" (UID: \"5cc57719-0da2-430d-8f48-7ef9d9d774fd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"
Jul 08 16:33:37.097738 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.097383    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh29d\" (UniqueName: \"kubernetes.io/projected/113bd2ab-b61a-41f1-a106-0336460bfae4-kube-api-access-wh29d\") pod \"service-ca-operator-5d46d9ffd4-67sxp\" (UID: \"113bd2ab-b61a-41f1-a106-0336460bfae4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.098115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.098080    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f26ps\" (UniqueName: \"kubernetes.io/projected/db950e2c-5b14-484d-941d-879cff4b3343-kube-api-access-f26ps\") pod \"insights-operator-5c847dcfc4-xh8sq\" (UID: \"db950e2c-5b14-484d-941d-879cff4b3343\") " pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.124631 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.124606    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"
Jul 08 16:33:37.140788 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.140767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"
Jul 08 16:33:37.167622 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.167594    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191335    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2tfp\" (UniqueName: \"kubernetes.io/projected/84b7f28c-4a45-41ad-af20-44ca3821d67f-kube-api-access-t2tfp\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqnm8\" (UniqueName: \"kubernetes.io/projected/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-kube-api-access-qqnm8\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8svd\" (UniqueName: \"kubernetes.io/projected/12d88f79-f370-45df-9935-944f43c1c1d7-kube-api-access-h8svd\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191468    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12d88f79-f370-45df-9935-944f43c1c1d7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gljf\" (UniqueName: \"kubernetes.io/projected/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-kube-api-access-4gljf\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b7f28c-4a45-41ad-af20-44ca3821d67f-config\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-default-certificate\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-stats-auth\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12d88f79-f370-45df-9935-944f43c1c1d7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.192892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84b7f28c-4a45-41ad-af20-44ca3821d67f-serving-cert\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.193619 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b7f28c-4a45-41ad-af20-44ca3821d67f-trusted-ca\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.193619 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.191721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.193619 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.193609    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12d88f79-f370-45df-9935-944f43c1c1d7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.196184 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.196019    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-56gsq" event={"ID":"7130273a-98c0-46d9-b77e-ed40a3bb73be","Type":"ContainerStarted","Data":"3d31bf67e6be39717910d375c23e97350d9105111850d23afab771bde5112dfc"}
Jul 08 16:33:37.197267 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.197212    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12d88f79-f370-45df-9935-944f43c1c1d7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.202787 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.202762    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8svd\" (UniqueName: \"kubernetes.io/projected/12d88f79-f370-45df-9935-944f43c1c1d7-kube-api-access-h8svd\") pod \"kube-storage-version-migrator-operator-846bf668f7-fqgl7\" (UID: \"12d88f79-f370-45df-9935-944f43c1c1d7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.213390 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.213226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:37.234171 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.233946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84b7f28c-4a45-41ad-af20-44ca3821d67f-serving-cert\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b7f28c-4a45-41ad-af20-44ca3821d67f-trusted-ca\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2tfp\" (UniqueName: \"kubernetes.io/projected/84b7f28c-4a45-41ad-af20-44ca3821d67f-kube-api-access-t2tfp\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqnm8\" (UniqueName: \"kubernetes.io/projected/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-kube-api-access-qqnm8\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gljf\" (UniqueName: \"kubernetes.io/projected/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-kube-api-access-4gljf\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.292998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b7f28c-4a45-41ad-af20-44ca3821d67f-config\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.293024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-default-certificate\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.293050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-stats-auth\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.293558    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 16:33:37.295366 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.294599    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:37.794572858 +0000 UTC m=+38.356550233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : secret "router-metrics-certs-default" not found
Jul 08 16:33:37.296355 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.295646    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b7f28c-4a45-41ad-af20-44ca3821d67f-trusted-ca\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.296355 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.295797    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:37.795775268 +0000 UTC m=+38.357752642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : configmap references non-existent config key: service-ca.crt
Jul 08 16:33:37.296355 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.295883    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 16:33:37.296355 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.295926    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls podName:10bfbe49-9e90-4598-94c6-0aaa84b4ad1c nodeName:}" failed. No retries permitted until 2026-07-08 16:33:37.795912933 +0000 UTC m=+38.357890297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qfp4m" (UID: "10bfbe49-9e90-4598-94c6-0aaa84b4ad1c") : secret "samples-operator-tls" not found
Jul 08 16:33:37.296355 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.296270    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b7f28c-4a45-41ad-af20-44ca3821d67f-config\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.296563 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.296500    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-stats-auth\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.297576 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.297521    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84b7f28c-4a45-41ad-af20-44ca3821d67f-serving-cert\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.301375 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.301302    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-default-certificate\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.307098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.307043    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqnm8\" (UniqueName: \"kubernetes.io/projected/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-kube-api-access-qqnm8\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.312167 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.307745    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gljf\" (UniqueName: \"kubernetes.io/projected/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-kube-api-access-4gljf\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.312508 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.312400    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2tfp\" (UniqueName: \"kubernetes.io/projected/84b7f28c-4a45-41ad-af20-44ca3821d67f-kube-api-access-t2tfp\") pod \"console-operator-68b757fb67-tnz7d\" (UID: \"84b7f28c-4a45-41ad-af20-44ca3821d67f\") " pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.316074 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.316017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:37.341862 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.341182    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq"]
Jul 08 16:33:37.347933 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:37.347886    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc57719_0da2_430d_8f48_7ef9d9d774fd.slice/crio-faf2ee20812d68cfe7185ea22d47fba7383d5b848d11d0ebc2aa0edaa052cd5b WatchSource:0}: Error finding container faf2ee20812d68cfe7185ea22d47fba7383d5b848d11d0ebc2aa0edaa052cd5b: Status 404 returned error can't find the container with id faf2ee20812d68cfe7185ea22d47fba7383d5b848d11d0ebc2aa0edaa052cd5b
Jul 08 16:33:37.364953 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.364724    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp"]
Jul 08 16:33:37.368261 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:37.368149    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113bd2ab_b61a_41f1_a106_0336460bfae4.slice/crio-268996473ef05360e891f0dfd3bd380b45c46742d10e3b0f43aba0b0a481a7f3 WatchSource:0}: Error finding container 268996473ef05360e891f0dfd3bd380b45c46742d10e3b0f43aba0b0a481a7f3: Status 404 returned error can't find the container with id 268996473ef05360e891f0dfd3bd380b45c46742d10e3b0f43aba0b0a481a7f3
Jul 08 16:33:37.378031 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.377714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:37.394865 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.394787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:37.395510 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.395094    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:37.395510 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.395165    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls podName:b45abeec-9848-44ca-b486-42f75a62b891 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:38.395145467 +0000 UTC m=+38.957122843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xgl2r" (UID: "b45abeec-9848-44ca-b486-42f75a62b891") : secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:37.410530 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.410044    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xh8sq"]
Jul 08 16:33:37.413484 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:37.413086    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb950e2c_5b14_484d_941d_879cff4b3343.slice/crio-1d6f94cc6224a4ec4c02ede274454bf695fc044a6dbd004337c1650b46b86d1e WatchSource:0}: Error finding container 1d6f94cc6224a4ec4c02ede274454bf695fc044a6dbd004337c1650b46b86d1e: Status 404 returned error can't find the container with id 1d6f94cc6224a4ec4c02ede274454bf695fc044a6dbd004337c1650b46b86d1e
Jul 08 16:33:37.457181 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.457151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7"]
Jul 08 16:33:37.464962 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:37.464006    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d88f79_f370_45df_9935_944f43c1c1d7.slice/crio-b6d883cc6d42cdca2da36e87c5d5cff8838bd5e6b7edf7e05d5660ad31b0cf58 WatchSource:0}: Error finding container b6d883cc6d42cdca2da36e87c5d5cff8838bd5e6b7edf7e05d5660ad31b0cf58: Status 404 returned error can't find the container with id b6d883cc6d42cdca2da36e87c5d5cff8838bd5e6b7edf7e05d5660ad31b0cf58
Jul 08 16:33:37.496743 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.496713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:37.496916 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.496896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:37.497377 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.497080    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 16:33:37.497377 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.497108    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 16:33:37.497377 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.497120    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db7d84484-znjz9: secret "image-registry-tls" not found
Jul 08 16:33:37.497377 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.497146    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls podName:539de09f-b3b6-4ed0-8e63-284856086968 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:39.497128058 +0000 UTC m=+40.059105440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls") pod "dns-default-vctc7" (UID: "539de09f-b3b6-4ed0-8e63-284856086968") : secret "dns-default-metrics-tls" not found
Jul 08 16:33:37.497377 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.497166    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls podName:894c4f16-253b-4e22-8fc7-1fe5aa158afc nodeName:}" failed. No retries permitted until 2026-07-08 16:33:39.497154254 +0000 UTC m=+40.059131629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls") pod "image-registry-6db7d84484-znjz9" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc") : secret "image-registry-tls" not found
Jul 08 16:33:37.569053 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.569032    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tnz7d"]
Jul 08 16:33:37.571107 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:37.571080    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b7f28c_4a45_41ad_af20_44ca3821d67f.slice/crio-d9f40695d205223c94d74b44b8659a148a7716ba0c57de823b41c970ca167aeb WatchSource:0}: Error finding container d9f40695d205223c94d74b44b8659a148a7716ba0c57de823b41c970ca167aeb: Status 404 returned error can't find the container with id d9f40695d205223c94d74b44b8659a148a7716ba0c57de823b41c970ca167aeb
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.800663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.800734    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:37.800773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.800854    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:38.80081358 +0000 UTC m=+39.362790959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : configmap references non-existent config key: service-ca.crt
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.800899    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.800936    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:38.800926244 +0000 UTC m=+39.362903611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : secret "router-metrics-certs-default" not found
Jul 08 16:33:37.800995 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.800943    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 16:33:37.801452 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:37.801031    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls podName:10bfbe49-9e90-4598-94c6-0aaa84b4ad1c nodeName:}" failed. No retries permitted until 2026-07-08 16:33:38.801013037 +0000 UTC m=+39.362990406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qfp4m" (UID: "10bfbe49-9e90-4598-94c6-0aaa84b4ad1c") : secret "samples-operator-tls" not found
Jul 08 16:33:38.204071 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.204034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" event={"ID":"84b7f28c-4a45-41ad-af20-44ca3821d67f","Type":"ContainerStarted","Data":"d9f40695d205223c94d74b44b8659a148a7716ba0c57de823b41c970ca167aeb"}
Jul 08 16:33:38.208070 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.208041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq" event={"ID":"db950e2c-5b14-484d-941d-879cff4b3343","Type":"ContainerStarted","Data":"1d6f94cc6224a4ec4c02ede274454bf695fc044a6dbd004337c1650b46b86d1e"}
Jul 08 16:33:38.210406 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.210380    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp" event={"ID":"113bd2ab-b61a-41f1-a106-0336460bfae4","Type":"ContainerStarted","Data":"268996473ef05360e891f0dfd3bd380b45c46742d10e3b0f43aba0b0a481a7f3"}
Jul 08 16:33:38.215518 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.215494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq" event={"ID":"5cc57719-0da2-430d-8f48-7ef9d9d774fd","Type":"ContainerStarted","Data":"faf2ee20812d68cfe7185ea22d47fba7383d5b848d11d0ebc2aa0edaa052cd5b"}
Jul 08 16:33:38.219924 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.219895    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7" event={"ID":"12d88f79-f370-45df-9935-944f43c1c1d7","Type":"ContainerStarted","Data":"b6d883cc6d42cdca2da36e87c5d5cff8838bd5e6b7edf7e05d5660ad31b0cf58"}
Jul 08 16:33:38.407225 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.406576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:38.407225 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.406780    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:38.407225 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.406862    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls podName:b45abeec-9848-44ca-b486-42f75a62b891 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:40.406842979 +0000 UTC m=+40.968820348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xgl2r" (UID: "b45abeec-9848-44ca-b486-42f75a62b891") : secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:38.811461 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.811424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:38.811648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.811496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:38.811648 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:38.811525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:38.811772 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.811749    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:40.811731872 +0000 UTC m=+41.373709233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : configmap references non-existent config key: service-ca.crt
Jul 08 16:33:38.812172 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.811968    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 16:33:38.812172 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.812025    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:40.812009892 +0000 UTC m=+41.373987278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : secret "router-metrics-certs-default" not found
Jul 08 16:33:38.812172 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.812022    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 16:33:38.812172 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:38.812090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls podName:10bfbe49-9e90-4598-94c6-0aaa84b4ad1c nodeName:}" failed. No retries permitted until 2026-07-08 16:33:40.812072489 +0000 UTC m=+41.374049852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qfp4m" (UID: "10bfbe49-9e90-4598-94c6-0aaa84b4ad1c") : secret "samples-operator-tls" not found
Jul 08 16:33:39.518437 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:39.518383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:39.519159 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:39.518475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:39.519159 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:39.518648    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 16:33:39.519159 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:39.518662    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db7d84484-znjz9: secret "image-registry-tls" not found
Jul 08 16:33:39.519159 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:39.518719    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls podName:894c4f16-253b-4e22-8fc7-1fe5aa158afc nodeName:}" failed. No retries permitted until 2026-07-08 16:33:43.518701577 +0000 UTC m=+44.080678945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls") pod "image-registry-6db7d84484-znjz9" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc") : secret "image-registry-tls" not found
Jul 08 16:33:39.519159 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:39.518932    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 16:33:39.519159 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:39.518990    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls podName:539de09f-b3b6-4ed0-8e63-284856086968 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:43.518975224 +0000 UTC m=+44.080952597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls") pod "dns-default-vctc7" (UID: "539de09f-b3b6-4ed0-8e63-284856086968") : secret "dns-default-metrics-tls" not found
Jul 08 16:33:40.394864 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:40.394628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:40.426158 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:40.426122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:40.426338 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.426291    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:40.426405 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.426347    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls podName:b45abeec-9848-44ca-b486-42f75a62b891 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:44.426332611 +0000 UTC m=+44.988309973 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xgl2r" (UID: "b45abeec-9848-44ca-b486-42f75a62b891") : secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:40.514088 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:40.514048    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:40.830229 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:40.830138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:40.830229 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:40.830203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:40.830614 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:40.830327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:40.830614 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.830331    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 16:33:40.830614 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.830398    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 16:33:40.830614 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.830470    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls podName:10bfbe49-9e90-4598-94c6-0aaa84b4ad1c nodeName:}" failed. No retries permitted until 2026-07-08 16:33:44.830448234 +0000 UTC m=+45.392425599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qfp4m" (UID: "10bfbe49-9e90-4598-94c6-0aaa84b4ad1c") : secret "samples-operator-tls" not found
Jul 08 16:33:40.830614 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.830491    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:44.830480145 +0000 UTC m=+45.392457507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : configmap references non-existent config key: service-ca.crt
Jul 08 16:33:40.830614 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:40.830544    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:44.830526446 +0000 UTC m=+45.392503822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : secret "router-metrics-certs-default" not found
Jul 08 16:33:41.922259 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:41.922227    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"]
Jul 08 16:33:41.937280 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:41.937255    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"]
Jul 08 16:33:41.937389 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:41.937349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"
Jul 08 16:33:41.939854 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:41.939817    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-z9vtn\""
Jul 08 16:33:42.042382 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:42.042349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4dq7\" (UniqueName: \"kubernetes.io/projected/cb132fbd-4b29-461b-b48b-7536187aee73-kube-api-access-n4dq7\") pod \"network-check-source-989f8b8dc-9lf9x\" (UID: \"cb132fbd-4b29-461b-b48b-7536187aee73\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"
Jul 08 16:33:42.143488 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:42.143444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4dq7\" (UniqueName: \"kubernetes.io/projected/cb132fbd-4b29-461b-b48b-7536187aee73-kube-api-access-n4dq7\") pod \"network-check-source-989f8b8dc-9lf9x\" (UID: \"cb132fbd-4b29-461b-b48b-7536187aee73\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"
Jul 08 16:33:42.153187 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:42.153158    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4dq7\" (UniqueName: \"kubernetes.io/projected/cb132fbd-4b29-461b-b48b-7536187aee73-kube-api-access-n4dq7\") pod \"network-check-source-989f8b8dc-9lf9x\" (UID: \"cb132fbd-4b29-461b-b48b-7536187aee73\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"
Jul 08 16:33:42.247911 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:42.247838    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"
Jul 08 16:33:43.554611 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:43.554577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:43.555069 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:43.554666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:43.555069 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:43.554768    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 16:33:43.555069 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:43.554815    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 16:33:43.555069 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:43.554848    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db7d84484-znjz9: secret "image-registry-tls" not found
Jul 08 16:33:43.555069 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:43.554865    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls podName:539de09f-b3b6-4ed0-8e63-284856086968 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:51.554843425 +0000 UTC m=+52.116820799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls") pod "dns-default-vctc7" (UID: "539de09f-b3b6-4ed0-8e63-284856086968") : secret "dns-default-metrics-tls" not found
Jul 08 16:33:43.555069 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:43.554916    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls podName:894c4f16-253b-4e22-8fc7-1fe5aa158afc nodeName:}" failed. No retries permitted until 2026-07-08 16:33:51.554886389 +0000 UTC m=+52.116863750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls") pod "image-registry-6db7d84484-znjz9" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc") : secret "image-registry-tls" not found
Jul 08 16:33:44.463639 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:44.463599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:44.463839 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.463757    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:44.463914 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.463851    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls podName:b45abeec-9848-44ca-b486-42f75a62b891 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:52.463812775 +0000 UTC m=+53.025790159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xgl2r" (UID: "b45abeec-9848-44ca-b486-42f75a62b891") : secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:44.868178 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:44.868146    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:44.868194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:44.868217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.868316    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.868322    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:52.868306985 +0000 UTC m=+53.430284347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : configmap references non-existent config key: service-ca.crt
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.868325    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.868357    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:52.868345787 +0000 UTC m=+53.430323149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : secret "router-metrics-certs-default" not found
Jul 08 16:33:44.868631 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:44.868370    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls podName:10bfbe49-9e90-4598-94c6-0aaa84b4ad1c nodeName:}" failed. No retries permitted until 2026-07-08 16:33:52.868361141 +0000 UTC m=+53.430338510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qfp4m" (UID: "10bfbe49-9e90-4598-94c6-0aaa84b4ad1c") : secret "samples-operator-tls" not found
Jul 08 16:33:45.358494 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:45.358465    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x"]
Jul 08 16:33:45.362651 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:45.362622    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb132fbd_4b29_461b_b48b_7536187aee73.slice/crio-1ec332119eaf1b798c09de44eb2c7a52f7e0d01f820b28d8954851bcdf186611 WatchSource:0}: Error finding container 1ec332119eaf1b798c09de44eb2c7a52f7e0d01f820b28d8954851bcdf186611: Status 404 returned error can't find the container with id 1ec332119eaf1b798c09de44eb2c7a52f7e0d01f820b28d8954851bcdf186611
Jul 08 16:33:46.242392 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.242365    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/0.log"
Jul 08 16:33:46.242875 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.242407    2562 generic.go:356] "Generic (PLEG): container finished" podID="84b7f28c-4a45-41ad-af20-44ca3821d67f" containerID="acd5055fd1978a350abf4c3737ce3c7662e7ba832190788fd93df7f852aac7fe" exitCode=255
Jul 08 16:33:46.242875 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.242447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" event={"ID":"84b7f28c-4a45-41ad-af20-44ca3821d67f","Type":"ContainerDied","Data":"acd5055fd1978a350abf4c3737ce3c7662e7ba832190788fd93df7f852aac7fe"}
Jul 08 16:33:46.242875 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.242734    2562 scope.go:117] "RemoveContainer" containerID="acd5055fd1978a350abf4c3737ce3c7662e7ba832190788fd93df7f852aac7fe"
Jul 08 16:33:46.244164 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.244129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-56gsq" event={"ID":"7130273a-98c0-46d9-b77e-ed40a3bb73be","Type":"ContainerStarted","Data":"bc87facb39b0e9c66e5c9e6f1d62332d271975f83eee9409bc8a6c21981528a6"}
Jul 08 16:33:46.245753 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.245729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq" event={"ID":"db950e2c-5b14-484d-941d-879cff4b3343","Type":"ContainerStarted","Data":"5ddbc877005d1229559e9c76c4b0cb3074f8f3ec61b8f5f70b5600a42639cf09"}
Jul 08 16:33:46.247349 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.247322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp" event={"ID":"113bd2ab-b61a-41f1-a106-0336460bfae4","Type":"ContainerStarted","Data":"723c4f599f6ccb68ca30f2a7c9b94cddc63c355d691ca949729d20c2cc092001"}
Jul 08 16:33:46.248713 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.248693    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq" event={"ID":"5cc57719-0da2-430d-8f48-7ef9d9d774fd","Type":"ContainerStarted","Data":"01ecbafd7c3f2cd2639a88eb0eb9a4f4a93bcc4ce1a6d67a5fd0a5fde526f1f9"}
Jul 08 16:33:46.251481 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.251462    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f" containerID="2bfae60342de000132e7e5b8daf771ee48dd35a842121a5986566975cd668377" exitCode=0
Jul 08 16:33:46.251559 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.251531    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerDied","Data":"2bfae60342de000132e7e5b8daf771ee48dd35a842121a5986566975cd668377"}
Jul 08 16:33:46.253073 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.253044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7" event={"ID":"12d88f79-f370-45df-9935-944f43c1c1d7","Type":"ContainerStarted","Data":"b990693d71607dead7278a5e502ded6a24bde89be7e87cc262f3bc9a36bc5291"}
Jul 08 16:33:46.254552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.254533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x" event={"ID":"cb132fbd-4b29-461b-b48b-7536187aee73","Type":"ContainerStarted","Data":"1ec332119eaf1b798c09de44eb2c7a52f7e0d01f820b28d8954851bcdf186611"}
Jul 08 16:33:46.277263 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.277059    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-56gsq" podStartSLOduration=2.2979349 podStartE2EDuration="11.277042169s" totalImagesPullingTime="8.979107269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:46.276421737 +0000 UTC m=+46.838399120" watchObservedRunningTime="2026-07-08 16:33:46.277042169 +0000 UTC m=+46.839019555"
Jul 08 16:33:46.317848 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.317753    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp" podStartSLOduration=2.46175211 podStartE2EDuration="10.317715354s" totalImagesPullingTime="7.855963244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:46.296709577 +0000 UTC m=+46.858686962" watchObservedRunningTime="2026-07-08 16:33:46.317715354 +0000 UTC m=+46.879708570"
Jul 08 16:33:46.318937 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.318454    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7" podStartSLOduration=2.5553146079999998 podStartE2EDuration="10.318381572s" totalImagesPullingTime="7.763066964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:46.316517824 +0000 UTC m=+46.878495208" watchObservedRunningTime="2026-07-08 16:33:46.318381572 +0000 UTC m=+46.880358957"
Jul 08 16:33:46.362700 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.362330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq" podStartSLOduration=2.545426728 podStartE2EDuration="10.362308529s" totalImagesPullingTime="7.816881801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:46.360906606 +0000 UTC m=+46.922884005" watchObservedRunningTime="2026-07-08 16:33:46.362308529 +0000 UTC m=+46.924285914"
Jul 08 16:33:46.379742 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.378475    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-29gdq" podStartSLOduration=2.502699455 podStartE2EDuration="10.378456272s" totalImagesPullingTime="7.875756817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:46.378135842 +0000 UTC m=+46.940113220" watchObservedRunningTime="2026-07-08 16:33:46.378456272 +0000 UTC m=+46.940433658"
Jul 08 16:33:46.543687 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.543654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:46.604252 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.604216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:46.898140 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.898106    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"]
Jul 08 16:33:46.930656 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.930618    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"]
Jul 08 16:33:46.930849 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.930751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:46.933293 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.933271    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 08 16:33:46.933411 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.933315    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 08 16:33:46.933554 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.933533    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dwmx2\""
Jul 08 16:33:46.989897 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.989872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4342e11-7942-4b75-b0c5-d1b808b90bd2-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:46.990032 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:46.989917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:47.028174 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.028147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.090470 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.090439    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4342e11-7942-4b75-b0c5-d1b808b90bd2-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:47.090571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.090507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:47.090674 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.090656    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 16:33:47.090754 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.090743    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert podName:b4342e11-7942-4b75-b0c5-d1b808b90bd2 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:47.590715207 +0000 UTC m=+48.152692572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-t88rq" (UID: "b4342e11-7942-4b75-b0c5-d1b808b90bd2") : secret "networking-console-plugin-cert" not found
Jul 08 16:33:47.091253 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.091233    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4342e11-7942-4b75-b0c5-d1b808b90bd2-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:47.100189 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.100164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.191342 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.191257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.265074 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.265043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.268341 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.268312    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f" containerID="8dfbf0efde55f1b94bc622fa959880f3190ccb12554bd9097bb02c8f93872907" exitCode=0
Jul 08 16:33:47.268569 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.268376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerDied","Data":"8dfbf0efde55f1b94bc622fa959880f3190ccb12554bd9097bb02c8f93872907"}
Jul 08 16:33:47.271059 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.271034    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/1.log"
Jul 08 16:33:47.271411 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.271398    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/0.log"
Jul 08 16:33:47.271476 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.271427    2562 generic.go:356] "Generic (PLEG): container finished" podID="84b7f28c-4a45-41ad-af20-44ca3821d67f" containerID="af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902" exitCode=255
Jul 08 16:33:47.271510 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.271485    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" event={"ID":"84b7f28c-4a45-41ad-af20-44ca3821d67f","Type":"ContainerDied","Data":"af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902"}
Jul 08 16:33:47.271558 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.271517    2562 scope.go:117] "RemoveContainer" containerID="acd5055fd1978a350abf4c3737ce3c7662e7ba832190788fd93df7f852aac7fe"
Jul 08 16:33:47.271713 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.271693    2562 scope.go:117] "RemoveContainer" containerID="af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902"
Jul 08 16:33:47.271905 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.271882    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tnz7d_openshift-console-operator(84b7f28c-4a45-41ad-af20-44ca3821d67f)\"" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podUID="84b7f28c-4a45-41ad-af20-44ca3821d67f"
Jul 08 16:33:47.322844 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.322433    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tcvsm"]
Jul 08 16:33:47.357362 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.357332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tcvsm"]
Jul 08 16:33:47.357526 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.357458    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.360202 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.360168    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 16:33:47.360202 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.360195    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 16:33:47.360410 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.360283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mg5k6\""
Jul 08 16:33:47.378858 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.378812    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:47.378967 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.378875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:33:47.392533 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.392511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.392810 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.392784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92d89032-2986-42a8-af24-0944878393fa-crio-socket\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.393260 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.393224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7fnw\" (UniqueName: \"kubernetes.io/projected/92d89032-2986-42a8-af24-0944878393fa-kube-api-access-t7fnw\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.393394 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.393369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d89032-2986-42a8-af24-0944878393fa-data-volume\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.393473 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.393444    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.399240 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.399217    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"]
Jul 08 16:33:47.405415 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.405390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.415630 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.415168    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"]
Jul 08 16:33:47.415630 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.415278    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"
Jul 08 16:33:47.418169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.418145    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 08 16:33:47.418433 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.418414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-s2rfb\""
Jul 08 16:33:47.418690 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.418672    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 08 16:33:47.494358 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92d89032-2986-42a8-af24-0944878393fa-crio-socket\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494358 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7fnw\" (UniqueName: \"kubernetes.io/projected/92d89032-2986-42a8-af24-0944878393fa-kube-api-access-t7fnw\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d89032-2986-42a8-af24-0944878393fa-data-volume\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494459    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxndc\" (UniqueName: \"kubernetes.io/projected/65d6ec16-0be4-47a2-9282-6961647a9ff1-kube-api-access-nxndc\") pod \"migrator-d57d58cc9-nps8b\" (UID: \"65d6ec16-0be4-47a2-9282-6961647a9ff1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"
Jul 08 16:33:47.494555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494506    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92d89032-2986-42a8-af24-0944878393fa-crio-socket\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494555 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.494549    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:47.494742 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.494601    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:47.994585722 +0000 UTC m=+48.556563083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:47.494742 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494742 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.494688    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 16:33:47.494742 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.494712    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d89032-2986-42a8-af24-0944878393fa-data-volume\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.494742 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.494716    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:47.994708608 +0000 UTC m=+48.556685970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : configmap "kube-rbac-proxy" not found
Jul 08 16:33:47.503482 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.503453    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7fnw\" (UniqueName: \"kubernetes.io/projected/92d89032-2986-42a8-af24-0944878393fa-kube-api-access-t7fnw\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:47.510441 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.510397    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.595560 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.595534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:47.595704 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.595601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxndc\" (UniqueName: \"kubernetes.io/projected/65d6ec16-0be4-47a2-9282-6961647a9ff1-kube-api-access-nxndc\") pod \"migrator-d57d58cc9-nps8b\" (UID: \"65d6ec16-0be4-47a2-9282-6961647a9ff1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"
Jul 08 16:33:47.595779 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.595758    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 16:33:47.595899 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.595857    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert podName:b4342e11-7942-4b75-b0c5-d1b808b90bd2 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:48.595816295 +0000 UTC m=+49.157793678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-t88rq" (UID: "b4342e11-7942-4b75-b0c5-d1b808b90bd2") : secret "networking-console-plugin-cert" not found
Jul 08 16:33:47.603979 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.603953    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxndc\" (UniqueName: \"kubernetes.io/projected/65d6ec16-0be4-47a2-9282-6961647a9ff1-kube-api-access-nxndc\") pod \"migrator-d57d58cc9-nps8b\" (UID: \"65d6ec16-0be4-47a2-9282-6961647a9ff1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"
Jul 08 16:33:47.612755 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.612730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.713981 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.713949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.726902 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.726877    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"
Jul 08 16:33:47.812951 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.812923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:47.865390 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.865265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b"]
Jul 08 16:33:47.867874 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:47.867849    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d6ec16_0be4_47a2_9282_6961647a9ff1.slice/crio-63d8418e039df8264a10801e0bfd9ef7d096f3ce4558742482926afef36ea707 WatchSource:0}: Error finding container 63d8418e039df8264a10801e0bfd9ef7d096f3ce4558742482926afef36ea707: Status 404 returned error can't find the container with id 63d8418e039df8264a10801e0bfd9ef7d096f3ce4558742482926afef36ea707
Jul 08 16:33:47.999848 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.999741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:48.000015 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:47.999904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:48.000015 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.999930    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:48.000015 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:47.999991    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 16:33:48.000015 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:48.000011    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:48.9999902 +0000 UTC m=+49.561967564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:48.000178 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:48.000051    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:49.000034183 +0000 UTC m=+49.562011573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : configmap "kube-rbac-proxy" not found
Jul 08 16:33:48.280140 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.280025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" event={"ID":"6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f","Type":"ContainerStarted","Data":"3d923c8beba8b8ce4ada28427ed6a0ba8ff73b86ced2cf3de06e9986face5ef7"}
Jul 08 16:33:48.281895 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.281871    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/1.log"
Jul 08 16:33:48.282224 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.282204    2562 scope.go:117] "RemoveContainer" containerID="af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902"
Jul 08 16:33:48.282400 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:48.282374    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tnz7d_openshift-console-operator(84b7f28c-4a45-41ad-af20-44ca3821d67f)\"" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podUID="84b7f28c-4a45-41ad-af20-44ca3821d67f"
Jul 08 16:33:48.283607 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.283585    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b" event={"ID":"65d6ec16-0be4-47a2-9282-6961647a9ff1","Type":"ContainerStarted","Data":"63d8418e039df8264a10801e0bfd9ef7d096f3ce4558742482926afef36ea707"}
Jul 08 16:33:48.308886 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.308802    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sr2lk" podStartSLOduration=8.129062746 podStartE2EDuration="47.308783268s" totalImagesPullingTime="39.179720522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:48.306334455 +0000 UTC m=+48.868311841" watchObservedRunningTime="2026-07-08 16:33:48.308783268 +0000 UTC m=+48.870760660"
Jul 08 16:33:48.423145 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.423103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:48.490864 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.490810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:48.558022 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.557937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:48.605402 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.605363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:48.605550 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:48.605513    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 16:33:48.605612 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:48.605582    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert podName:b4342e11-7942-4b75-b0c5-d1b808b90bd2 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:50.605565509 +0000 UTC m=+51.167542870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-t88rq" (UID: "b4342e11-7942-4b75-b0c5-d1b808b90bd2") : secret "networking-console-plugin-cert" not found
Jul 08 16:33:48.713091 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:48.713050    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wzp4k_4f281f6c-3a96-491b-bb22-f8ae225f25b1/dns-node-resolver/0.log"
Jul 08 16:33:49.009068 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.009032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:49.009349 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.009126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:49.009349 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:49.009181    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:49.009349 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:49.009261    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 16:33:49.009349 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:49.009267    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:51.009244298 +0000 UTC m=+51.571221676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:49.009349 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:49.009342    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:51.009323626 +0000 UTC m=+51.571300990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : configmap "kube-rbac-proxy" not found
Jul 08 16:33:49.276068 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.275977    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-wpr8h"]
Jul 08 16:33:49.288677 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.288619    2562 scope.go:117] "RemoveContainer" containerID="af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902"
Jul 08 16:33:49.289133 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:49.288817    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tnz7d_openshift-console-operator(84b7f28c-4a45-41ad-af20-44ca3821d67f)\"" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podUID="84b7f28c-4a45-41ad-af20-44ca3821d67f"
Jul 08 16:33:49.299504 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.299479    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-wpr8h"]
Jul 08 16:33:49.299636 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.299596    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.302308 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.302281    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 08 16:33:49.302432 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.302395    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 08 16:33:49.302529 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.302504    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 08 16:33:49.303731 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.303709    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9hhkh\""
Jul 08 16:33:49.303808 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.303711    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 08 16:33:49.313693 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.313675    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cpggg_ed690582-0a9e-4069-892c-78c07ea8066a/node-ca/0.log"
Jul 08 16:33:49.338920 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.338888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:49.413048 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.413006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktbh8\" (UniqueName: \"kubernetes.io/projected/248838fd-e4fd-46cf-bcfa-e6ac03e58113-kube-api-access-ktbh8\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.413242 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.413176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/248838fd-e4fd-46cf-bcfa-e6ac03e58113-signing-key\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.413242 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.413215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/248838fd-e4fd-46cf-bcfa-e6ac03e58113-signing-cabundle\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.449639 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.449603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:49.514105 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.514060    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktbh8\" (UniqueName: \"kubernetes.io/projected/248838fd-e4fd-46cf-bcfa-e6ac03e58113-kube-api-access-ktbh8\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.514315 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.514139    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/248838fd-e4fd-46cf-bcfa-e6ac03e58113-signing-key\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.514315 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.514166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/248838fd-e4fd-46cf-bcfa-e6ac03e58113-signing-cabundle\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.514919 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.514895    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/248838fd-e4fd-46cf-bcfa-e6ac03e58113-signing-cabundle\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.516462 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.516434    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/248838fd-e4fd-46cf-bcfa-e6ac03e58113-signing-key\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.527966 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.527890    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktbh8\" (UniqueName: \"kubernetes.io/projected/248838fd-e4fd-46cf-bcfa-e6ac03e58113-kube-api-access-ktbh8\") pod \"service-ca-758f599c5-wpr8h\" (UID: \"248838fd-e4fd-46cf-bcfa-e6ac03e58113\") " pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:49.610993 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:49.610960    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-wpr8h"
Jul 08 16:33:50.113752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:50.113714    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-56gsq_7130273a-98c0-46d9-b77e-ed40a3bb73be/serve-healthcheck-canary/0.log"
Jul 08 16:33:50.408520 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:50.408401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:50.624973 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:50.624943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:50.625126 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:50.625113    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 16:33:50.625214 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:50.625203    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert podName:b4342e11-7942-4b75-b0c5-d1b808b90bd2 nodeName:}" failed. No retries permitted until 2026-07-08 16:33:54.625185723 +0000 UTC m=+55.187163382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-t88rq" (UID: "b4342e11-7942-4b75-b0c5-d1b808b90bd2") : secret "networking-console-plugin-cert" not found
Jul 08 16:33:50.797259 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:50.797219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:50.847251 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:50.847196    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-wpr8h"]
Jul 08 16:33:50.856155 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:33:50.856109    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248838fd_e4fd_46cf_bcfa_e6ac03e58113.slice/crio-3bedf1ffbf18d7eb86dd4048454d972c0bb718f5a1cce3731a942df2ba899dfa WatchSource:0}: Error finding container 3bedf1ffbf18d7eb86dd4048454d972c0bb718f5a1cce3731a942df2ba899dfa: Status 404 returned error can't find the container with id 3bedf1ffbf18d7eb86dd4048454d972c0bb718f5a1cce3731a942df2ba899dfa
Jul 08 16:33:51.028605 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.028529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:51.028767 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.028628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:51.028767 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.028707    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:51.028876 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.028789    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:33:55.028769445 +0000 UTC m=+55.590746810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:51.029268 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.029245    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92d89032-2986-42a8-af24-0944878393fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:51.296508 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.296422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x" event={"ID":"cb132fbd-4b29-461b-b48b-7536187aee73","Type":"ContainerStarted","Data":"67c206f2c746bf56d16bca6378c6a212a15e924510a98475707ab2a8c7769cae"}
Jul 08 16:33:51.297945 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.297918    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-wpr8h" event={"ID":"248838fd-e4fd-46cf-bcfa-e6ac03e58113","Type":"ContainerStarted","Data":"6c56d8b0adadb6e1533ca338154684a948acad1fc5b569eba8a75d4ead4c86fc"}
Jul 08 16:33:51.298060 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.297962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-wpr8h" event={"ID":"248838fd-e4fd-46cf-bcfa-e6ac03e58113","Type":"ContainerStarted","Data":"3bedf1ffbf18d7eb86dd4048454d972c0bb718f5a1cce3731a942df2ba899dfa"}
Jul 08 16:33:51.299612 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.299588    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b" event={"ID":"65d6ec16-0be4-47a2-9282-6961647a9ff1","Type":"ContainerStarted","Data":"8cc665fef5f0835b939ac4cb6fbe2ec043de79cd228b1896f00f4476ccf8603b"}
Jul 08 16:33:51.299695 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.299630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b" event={"ID":"65d6ec16-0be4-47a2-9282-6961647a9ff1","Type":"ContainerStarted","Data":"094ae23e0ac1405ade54012698f4ee4583fc163327eb504a477b0f97e0d195a4"}
Jul 08 16:33:51.322222 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.322179    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-9lf9x" podStartSLOduration=4.967172089 podStartE2EDuration="10.322164201s" totalImagesPullingTime="5.354992112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:51.321878599 +0000 UTC m=+51.883855985" watchObservedRunningTime="2026-07-08 16:33:51.322164201 +0000 UTC m=+51.884141585"
Jul 08 16:33:51.344694 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.344654    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nps8b" podStartSLOduration=1.502457022 podStartE2EDuration="4.344640821s" totalImagesPullingTime="2.842183799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:51.343760949 +0000 UTC m=+51.905738332" watchObservedRunningTime="2026-07-08 16:33:51.344640821 +0000 UTC m=+51.906618205"
Jul 08 16:33:51.364137 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.364089    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-wpr8h" podStartSLOduration=2.364073313 podStartE2EDuration="2.364073313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:51.363406471 +0000 UTC m=+51.925383855" watchObservedRunningTime="2026-07-08 16:33:51.364073313 +0000 UTC m=+51.926050698"
Jul 08 16:33:51.634461 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.634419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:33:51.634975 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.634587    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 16:33:51.634975 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:51.634603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:33:51.634975 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.634666    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls podName:539de09f-b3b6-4ed0-8e63-284856086968 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:07.634645662 +0000 UTC m=+68.196623046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls") pod "dns-default-vctc7" (UID: "539de09f-b3b6-4ed0-8e63-284856086968") : secret "dns-default-metrics-tls" not found
Jul 08 16:33:51.634975 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.634729    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 16:33:51.634975 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.634742    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db7d84484-znjz9: secret "image-registry-tls" not found
Jul 08 16:33:51.634975 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:51.634791    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls podName:894c4f16-253b-4e22-8fc7-1fe5aa158afc nodeName:}" failed. No retries permitted until 2026-07-08 16:34:07.634777269 +0000 UTC m=+68.196754637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls") pod "image-registry-6db7d84484-znjz9" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc") : secret "image-registry-tls" not found
Jul 08 16:33:52.484334 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.484298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:52.542841 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.542790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:33:52.543112 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:52.543095    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:52.543180 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:52.543166    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls podName:b45abeec-9848-44ca-b486-42f75a62b891 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:08.543148333 +0000 UTC m=+69.105125699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xgl2r" (UID: "b45abeec-9848-44ca-b486-42f75a62b891") : secret "cluster-monitoring-operator-tls" not found
Jul 08 16:33:52.638940 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.638899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:52.738756 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.738671    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:52.839247 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.839219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:52.915699 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.915653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:52.946422 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.946391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:52.946574 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:52.946554    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 16:33:52.946637 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:52.946626    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:08.946608213 +0000 UTC m=+69.508585590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : secret "router-metrics-certs-default" not found
Jul 08 16:33:52.946704 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.946688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:33:52.946786 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.946763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:52.946940 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:52.946864    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle podName:bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:08.94684732 +0000 UTC m=+69.508824683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle") pod "router-default-7df5b6f649-fxv58" (UID: "bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3") : configmap references non-existent config key: service-ca.crt
Jul 08 16:33:52.949257 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:52.949235    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10bfbe49-9e90-4598-94c6-0aaa84b4ad1c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qfp4m\" (UID: \"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:53.235421 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:53.235390    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"
Jul 08 16:33:53.363210 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:53.363178    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m"]
Jul 08 16:33:53.381273 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:53.381246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:53.516290 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:53.516208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:53.782160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:53.782061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:54.311989 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:54.311954    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m" event={"ID":"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c","Type":"ContainerStarted","Data":"83f0e24fb50412e3e810a09b35401fffca6ba12cbf3a68b9a35fff7b93a8a1fe"}
Jul 08 16:33:54.378637 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:54.378597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:54.662485 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:54.662450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:33:54.662690 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:54.662626    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 16:33:54.662774 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:54.662705    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert podName:b4342e11-7942-4b75-b0c5-d1b808b90bd2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:02.662684363 +0000 UTC m=+63.224661730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-t88rq" (UID: "b4342e11-7942-4b75-b0c5-d1b808b90bd2") : secret "networking-console-plugin-cert" not found
Jul 08 16:33:54.976549 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:54.976470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:55.065988 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:55.065954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:33:55.066168 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:55.066094    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:55.066168 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:33:55.066152    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls podName:92d89032-2986-42a8-af24-0944878393fa nodeName:}" failed. No retries permitted until 2026-07-08 16:34:03.066137232 +0000 UTC m=+63.628114594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcvsm" (UID: "92d89032-2986-42a8-af24-0944878393fa") : secret "insights-runtime-extractor-tls" not found
Jul 08 16:33:55.166732 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:55.166688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:55.372640 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:55.372604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:55.568420 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:55.568288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:55.781638 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:55.781606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log" limit=1024
Jul 08 16:33:56.324164 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:56.324130    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m" event={"ID":"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c","Type":"ContainerStarted","Data":"e48500aede37262aca6be0f66496c1a9b4fa07c9f5855dd0a8bf504f44683ffa"}
Jul 08 16:33:56.324569 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:56.324179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m" event={"ID":"10bfbe49-9e90-4598-94c6-0aaa84b4ad1c","Type":"ContainerStarted","Data":"80b1f0b0be9bfe458a667b284887e8e7438440cc38bedd5a8c231c77416f6915"}
Jul 08 16:33:56.351592 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:33:56.351534    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qfp4m" podStartSLOduration=18.099554386 podStartE2EDuration="20.351515903s" totalImagesPullingTime="2.251961517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:33:56.350653636 +0000 UTC m=+56.912631022" watchObservedRunningTime="2026-07-08 16:33:56.351515903 +0000 UTC m=+56.913493291"
Jul 08 16:34:01.178933 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:01.178898    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gx2vd"
Jul 08 16:34:02.736005 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:02.735970    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:34:02.738313 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:02.738293    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4342e11-7942-4b75-b0c5-d1b808b90bd2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-t88rq\" (UID: \"b4342e11-7942-4b75-b0c5-d1b808b90bd2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:34:02.843184 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:02.843152    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dwmx2\""
Jul 08 16:34:02.851463 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:02.851438    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"
Jul 08 16:34:02.972779 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:02.972748    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-t88rq"]
Jul 08 16:34:02.975651 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:02.975622    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4342e11_7942_4b75_b0c5_d1b808b90bd2.slice/crio-7677a9ca56430091e6d54552509c99d125ff74214104ca71be73509d00f2f46c WatchSource:0}: Error finding container 7677a9ca56430091e6d54552509c99d125ff74214104ca71be73509d00f2f46c: Status 404 returned error can't find the container with id 7677a9ca56430091e6d54552509c99d125ff74214104ca71be73509d00f2f46c
Jul 08 16:34:03.023660 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.023596    2562 scope.go:117] "RemoveContainer" containerID="af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902"
Jul 08 16:34:03.139422 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.139391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:34:03.141450 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.141430    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92d89032-2986-42a8-af24-0944878393fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcvsm\" (UID: \"92d89032-2986-42a8-af24-0944878393fa\") " pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:34:03.273987 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.273912    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mg5k6\""
Jul 08 16:34:03.281914 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.281893    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tcvsm"
Jul 08 16:34:03.350722 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.350701    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:34:03.351141 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.351128    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/1.log"
Jul 08 16:34:03.351222 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.351162    2562 generic.go:356] "Generic (PLEG): container finished" podID="84b7f28c-4a45-41ad-af20-44ca3821d67f" containerID="2f4f428e88b26219aa6430ee6ea73d1d02f71250685fa6fd2ebf505e6d999053" exitCode=255
Jul 08 16:34:03.351269 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.351236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" event={"ID":"84b7f28c-4a45-41ad-af20-44ca3821d67f","Type":"ContainerDied","Data":"2f4f428e88b26219aa6430ee6ea73d1d02f71250685fa6fd2ebf505e6d999053"}
Jul 08 16:34:03.351337 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.351320    2562 scope.go:117] "RemoveContainer" containerID="af9f77b0b191c0dafd095b67ba313141d4a08e527af33d95f36dc87222e5c902"
Jul 08 16:34:03.351691 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.351673    2562 scope.go:117] "RemoveContainer" containerID="2f4f428e88b26219aa6430ee6ea73d1d02f71250685fa6fd2ebf505e6d999053"
Jul 08 16:34:03.351906 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:34:03.351881    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-tnz7d_openshift-console-operator(84b7f28c-4a45-41ad-af20-44ca3821d67f)\"" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podUID="84b7f28c-4a45-41ad-af20-44ca3821d67f"
Jul 08 16:34:03.352615 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.352590    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq" event={"ID":"b4342e11-7942-4b75-b0c5-d1b808b90bd2","Type":"ContainerStarted","Data":"7677a9ca56430091e6d54552509c99d125ff74214104ca71be73509d00f2f46c"}
Jul 08 16:34:03.415123 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:03.415090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tcvsm"]
Jul 08 16:34:03.417718 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:03.417694    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d89032_2986_42a8_af24_0944878393fa.slice/crio-25e4fb317404e5ac9831d16009dd3d3a2f77ed29ff4b2357859d2660b8af7daf WatchSource:0}: Error finding container 25e4fb317404e5ac9831d16009dd3d3a2f77ed29ff4b2357859d2660b8af7daf: Status 404 returned error can't find the container with id 25e4fb317404e5ac9831d16009dd3d3a2f77ed29ff4b2357859d2660b8af7daf
Jul 08 16:34:04.358991 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:04.358896    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcvsm" event={"ID":"92d89032-2986-42a8-af24-0944878393fa","Type":"ContainerStarted","Data":"25e4fb317404e5ac9831d16009dd3d3a2f77ed29ff4b2357859d2660b8af7daf"}
Jul 08 16:34:04.361449 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:04.361386    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:34:05.366761 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.366718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq" event={"ID":"b4342e11-7942-4b75-b0c5-d1b808b90bd2","Type":"ContainerStarted","Data":"8c3be193ffcfda594de0391ed86628cb85cf8e8ce2977a32a8bfe383f53c7bd5"}
Jul 08 16:34:05.383651 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.383604    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-t88rq" podStartSLOduration=17.818597021 podStartE2EDuration="19.383588417s" totalImagesPullingTime="1.564991396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:05.383428539 +0000 UTC m=+65.945405926" watchObservedRunningTime="2026-07-08 16:34:05.383588417 +0000 UTC m=+65.945565801"
Jul 08 16:34:05.664787 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.664699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:34:05.667393 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.667370    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 16:34:05.677871 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.677846    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e967c6f7-5197-4648-8ce2-cffcd788026d-original-pull-secret\") pod \"global-pull-secret-syncer-mc44p\" (UID: \"e967c6f7-5197-4648-8ce2-cffcd788026d\") " pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:34:05.765393 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.765359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:34:05.768203 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.768181    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 16:34:05.778578 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.778554    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce8909d-08fb-45cb-a131-394cd09252a9-metrics-certs\") pod \"network-metrics-daemon-br9fz\" (UID: \"cce8909d-08fb-45cb-a131-394cd09252a9\") " pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:34:05.861215 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.861174    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc44p"
Jul 08 16:34:05.866674 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.866645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:34:05.869370 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.869346    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgqcb\" (UniqueName: \"kubernetes.io/projected/f75e0ea9-22bb-495a-99bc-117ccfccde9b-kube-api-access-lgqcb\") pod \"network-check-target-64tq5\" (UID: \"f75e0ea9-22bb-495a-99bc-117ccfccde9b\") " pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:34:05.871968 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.871944    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s7kjq\""
Jul 08 16:34:05.879833 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.879809    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:34:05.889213 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.889191    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4cx7\""
Jul 08 16:34:05.897404 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:05.897387    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-br9fz"
Jul 08 16:34:07.244748 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.244668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-br9fz"]
Jul 08 16:34:07.245353 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:07.245326    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce8909d_08fb_45cb_a131_394cd09252a9.slice/crio-c51f271372639f05571bb46771b89b0f5722a083e8c0bcfa48fb176bd2924a82 WatchSource:0}: Error finding container c51f271372639f05571bb46771b89b0f5722a083e8c0bcfa48fb176bd2924a82: Status 404 returned error can't find the container with id c51f271372639f05571bb46771b89b0f5722a083e8c0bcfa48fb176bd2924a82
Jul 08 16:34:07.261853 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.261813    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-64tq5"]
Jul 08 16:34:07.264616 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:07.264571    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75e0ea9_22bb_495a_99bc_117ccfccde9b.slice/crio-0cda6fe4e808dbe6aab3c1d153a5e540759d0e5ef0d06a379aeeeb9c46ff449e WatchSource:0}: Error finding container 0cda6fe4e808dbe6aab3c1d153a5e540759d0e5ef0d06a379aeeeb9c46ff449e: Status 404 returned error can't find the container with id 0cda6fe4e808dbe6aab3c1d153a5e540759d0e5ef0d06a379aeeeb9c46ff449e
Jul 08 16:34:07.277731 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.277707    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mc44p"]
Jul 08 16:34:07.279611 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:07.279587    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode967c6f7_5197_4648_8ce2_cffcd788026d.slice/crio-7e5836f2202bc0b29de59661759a968e81a7bd1752c128eec83923ca83414296 WatchSource:0}: Error finding container 7e5836f2202bc0b29de59661759a968e81a7bd1752c128eec83923ca83414296: Status 404 returned error can't find the container with id 7e5836f2202bc0b29de59661759a968e81a7bd1752c128eec83923ca83414296
Jul 08 16:34:07.376427 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.376393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mc44p" event={"ID":"e967c6f7-5197-4648-8ce2-cffcd788026d","Type":"ContainerStarted","Data":"7e5836f2202bc0b29de59661759a968e81a7bd1752c128eec83923ca83414296"}
Jul 08 16:34:07.377459 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.377437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-br9fz" event={"ID":"cce8909d-08fb-45cb-a131-394cd09252a9","Type":"ContainerStarted","Data":"c51f271372639f05571bb46771b89b0f5722a083e8c0bcfa48fb176bd2924a82"}
Jul 08 16:34:07.378008 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.377989    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:34:07.378444 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.378411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:34:07.382092 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.379108    2562 scope.go:117] "RemoveContainer" containerID="2f4f428e88b26219aa6430ee6ea73d1d02f71250685fa6fd2ebf505e6d999053"
Jul 08 16:34:07.382092 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:34:07.379423    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-tnz7d_openshift-console-operator(84b7f28c-4a45-41ad-af20-44ca3821d67f)\"" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podUID="84b7f28c-4a45-41ad-af20-44ca3821d67f"
Jul 08 16:34:07.383204 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.383179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-64tq5" event={"ID":"f75e0ea9-22bb-495a-99bc-117ccfccde9b","Type":"ContainerStarted","Data":"2514f3c561d1241615dd9ef1297169a33293ba96d206eb233bb3318a30f36287"}
Jul 08 16:34:07.383300 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.383219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-64tq5" event={"ID":"f75e0ea9-22bb-495a-99bc-117ccfccde9b","Type":"ContainerStarted","Data":"0cda6fe4e808dbe6aab3c1d153a5e540759d0e5ef0d06a379aeeeb9c46ff449e"}
Jul 08 16:34:07.383300 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.383266    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:34:07.384555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.384537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcvsm" event={"ID":"92d89032-2986-42a8-af24-0944878393fa","Type":"ContainerStarted","Data":"c14268abc7810a742cfb516b51ab3e9eac9cfcce1238b13b5a47ea7e027bba7e"}
Jul 08 16:34:07.412584 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.412547    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-64tq5" podStartSLOduration=66.412534986 podStartE2EDuration="1m6.412534986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:07.411981335 +0000 UTC m=+67.973958719" watchObservedRunningTime="2026-07-08 16:34:07.412534986 +0000 UTC m=+67.974512371"
Jul 08 16:34:07.684788 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.684707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:34:07.684788 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.684783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:34:07.687415 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.687384    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/539de09f-b3b6-4ed0-8e63-284856086968-metrics-tls\") pod \"dns-default-vctc7\" (UID: \"539de09f-b3b6-4ed0-8e63-284856086968\") " pod="openshift-dns/dns-default-vctc7"
Jul 08 16:34:07.687518 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.687467    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"image-registry-6db7d84484-znjz9\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") " pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:34:07.827627 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.827592    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dvnjd\""
Jul 08 16:34:07.835147 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.835126    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:34:07.836794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.836774    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4gk8\""
Jul 08 16:34:07.845125 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:07.845102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vctc7"
Jul 08 16:34:08.004188 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.004156    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6db7d84484-znjz9"]
Jul 08 16:34:08.020872 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.020850    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vctc7"]
Jul 08 16:34:08.144901 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:08.144863    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894c4f16_253b_4e22_8fc7_1fe5aa158afc.slice/crio-67f1ff3f9a0e354db2f7bec8da7730e1cbe75600e41ee67148ccd06f9bc0aba0 WatchSource:0}: Error finding container 67f1ff3f9a0e354db2f7bec8da7730e1cbe75600e41ee67148ccd06f9bc0aba0: Status 404 returned error can't find the container with id 67f1ff3f9a0e354db2f7bec8da7730e1cbe75600e41ee67148ccd06f9bc0aba0
Jul 08 16:34:08.146024 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:08.145805    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539de09f_b3b6_4ed0_8e63_284856086968.slice/crio-f0b41c79027c2f8ffd29c2faba13536e6a4e513621cc0234bc7db187db78ad0d WatchSource:0}: Error finding container f0b41c79027c2f8ffd29c2faba13536e6a4e513621cc0234bc7db187db78ad0d: Status 404 returned error can't find the container with id f0b41c79027c2f8ffd29c2faba13536e6a4e513621cc0234bc7db187db78ad0d
Jul 08 16:34:08.391713 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.391686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-br9fz" event={"ID":"cce8909d-08fb-45cb-a131-394cd09252a9","Type":"ContainerStarted","Data":"e3afee42b7234d378250d42e6c85d1e557201baba7ba92ebc13a343736cef15e"}
Jul 08 16:34:08.392729 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.392697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vctc7" event={"ID":"539de09f-b3b6-4ed0-8e63-284856086968","Type":"ContainerStarted","Data":"f0b41c79027c2f8ffd29c2faba13536e6a4e513621cc0234bc7db187db78ad0d"}
Jul 08 16:34:08.394866 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.394730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcvsm" event={"ID":"92d89032-2986-42a8-af24-0944878393fa","Type":"ContainerStarted","Data":"a3b02b7610acca97d9a413de70b0516404e216e1262bf93fa4bd90e181563db3"}
Jul 08 16:34:08.397253 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.397021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" event={"ID":"894c4f16-253b-4e22-8fc7-1fe5aa158afc","Type":"ContainerStarted","Data":"a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021"}
Jul 08 16:34:08.397253 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.397062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" event={"ID":"894c4f16-253b-4e22-8fc7-1fe5aa158afc","Type":"ContainerStarted","Data":"67f1ff3f9a0e354db2f7bec8da7730e1cbe75600e41ee67148ccd06f9bc0aba0"}
Jul 08 16:34:08.417344 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.417287    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" podStartSLOduration=67.417268623 podStartE2EDuration="1m7.417268623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:08.416505965 +0000 UTC m=+68.978483351" watchObservedRunningTime="2026-07-08 16:34:08.417268623 +0000 UTC m=+68.979246010"
Jul 08 16:34:08.591632 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.591597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:34:08.594426 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.594402    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b45abeec-9848-44ca-b486-42f75a62b891-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xgl2r\" (UID: \"b45abeec-9848-44ca-b486-42f75a62b891\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:34:08.785319 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.785286    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-77lg6\""
Jul 08 16:34:08.793281 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.793208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"
Jul 08 16:34:08.952897 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.952786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r"]
Jul 08 16:34:08.957501 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:08.956319    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45abeec_9848_44ca_b486_42f75a62b891.slice/crio-ec5c907b19124658f9b38ab3877a7923823a7d7b25337788b618ed91f983c0dd WatchSource:0}: Error finding container ec5c907b19124658f9b38ab3877a7923823a7d7b25337788b618ed91f983c0dd: Status 404 returned error can't find the container with id ec5c907b19124658f9b38ab3877a7923823a7d7b25337788b618ed91f983c0dd
Jul 08 16:34:08.996375 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.995215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:08.996375 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.995279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:08.996375 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:08.996328    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-service-ca-bundle\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:09.001228 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.001202    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3-metrics-certs\") pod \"router-default-7df5b6f649-fxv58\" (UID: \"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3\") " pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:09.215207 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.215125    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-27cr8\""
Jul 08 16:34:09.222992 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.222968    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:09.396860 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.396787    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7df5b6f649-fxv58"]
Jul 08 16:34:09.400643 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:09.400613    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda78aa0_0f42_40e1_bf1f_43d40bf5b0b3.slice/crio-6afa278b5654bcb077c26b3a724d368d4bdebc5169b4d7e54a1c9eb5dad7ccb3 WatchSource:0}: Error finding container 6afa278b5654bcb077c26b3a724d368d4bdebc5169b4d7e54a1c9eb5dad7ccb3: Status 404 returned error can't find the container with id 6afa278b5654bcb077c26b3a724d368d4bdebc5169b4d7e54a1c9eb5dad7ccb3
Jul 08 16:34:09.403694 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.403497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-br9fz" event={"ID":"cce8909d-08fb-45cb-a131-394cd09252a9","Type":"ContainerStarted","Data":"5916bfe0cd3d19606bf7bc8bc2974c8db1d8fead88f701b7d6a0f6e42b2ab545"}
Jul 08 16:34:09.404845 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.404787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r" event={"ID":"b45abeec-9848-44ca-b486-42f75a62b891","Type":"ContainerStarted","Data":"ec5c907b19124658f9b38ab3877a7923823a7d7b25337788b618ed91f983c0dd"}
Jul 08 16:34:09.406192 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.406144    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:34:09.423183 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:09.423126    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-br9fz" podStartSLOduration=67.410677941 podStartE2EDuration="1m8.423085722s" totalImagesPullingTime="1.012407781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:09.421553334 +0000 UTC m=+69.983530720" watchObservedRunningTime="2026-07-08 16:34:09.423085722 +0000 UTC m=+69.985063133"
Jul 08 16:34:10.411814 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:10.411762    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7df5b6f649-fxv58" event={"ID":"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3","Type":"ContainerStarted","Data":"bc8b4ad9f177eef0d8cd7838daae2301cd5da00527b8d76911a518011922e61c"}
Jul 08 16:34:10.412378 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:10.411875    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7df5b6f649-fxv58" event={"ID":"bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3","Type":"ContainerStarted","Data":"6afa278b5654bcb077c26b3a724d368d4bdebc5169b4d7e54a1c9eb5dad7ccb3"}
Jul 08 16:34:10.433198 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:10.433141    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7df5b6f649-fxv58" podStartSLOduration=34.433122949 podStartE2EDuration="34.433122949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:10.432186057 +0000 UTC m=+70.994163445" watchObservedRunningTime="2026-07-08 16:34:10.433122949 +0000 UTC m=+70.995100333"
Jul 08 16:34:11.224050 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:11.223912    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:11.227088 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:11.227061    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:11.416321 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:11.416284    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:11.417743 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:11.417721    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7df5b6f649-fxv58"
Jul 08 16:34:14.436628 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:14.436585    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcvsm" event={"ID":"92d89032-2986-42a8-af24-0944878393fa","Type":"ContainerStarted","Data":"103748ad366d868552d2a678ae80562b00c96ce3731c82f3a0a0fa545253ae2d"}
Jul 08 16:34:14.470600 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:14.470556    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tcvsm" podStartSLOduration=16.792502107 podStartE2EDuration="27.47054118s" totalImagesPullingTime="10.678039073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:14.469554366 +0000 UTC m=+75.031531772" watchObservedRunningTime="2026-07-08 16:34:14.47054118 +0000 UTC m=+75.032518564"
Jul 08 16:34:14.980372 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:14.980334    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"]
Jul 08 16:34:15.002713 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.002685    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"]
Jul 08 16:34:15.002850 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.002801    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:15.005190 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.005169    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 08 16:34:15.005663 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.005643    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cqvdl\""
Jul 08 16:34:15.145150 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.145114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6ded0201-533f-4b2b-b270-9605778b186c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jf9ck\" (UID: \"6ded0201-533f-4b2b-b270-9605778b186c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:15.246046 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.245966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6ded0201-533f-4b2b-b270-9605778b186c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jf9ck\" (UID: \"6ded0201-533f-4b2b-b270-9605778b186c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:15.248371 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.248353    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6ded0201-533f-4b2b-b270-9605778b186c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jf9ck\" (UID: \"6ded0201-533f-4b2b-b270-9605778b186c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:15.312498 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.312470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:15.442510 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.442478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vctc7" event={"ID":"539de09f-b3b6-4ed0-8e63-284856086968","Type":"ContainerStarted","Data":"6d992049fbe0130981dfd814efb87eded40b892925f75985b8b448ee033cc018"}
Jul 08 16:34:15.442943 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.442534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vctc7" event={"ID":"539de09f-b3b6-4ed0-8e63-284856086968","Type":"ContainerStarted","Data":"f2f00d695217d25e960ebd79592f15472c823b0ef2fd271d0ccf2abefc9cb586"}
Jul 08 16:34:15.442943 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.442576    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vctc7"
Jul 08 16:34:15.444123 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.444098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r" event={"ID":"b45abeec-9848-44ca-b486-42f75a62b891","Type":"ContainerStarted","Data":"a6c99266d0f02ea6aba9f77c640c6e62c091deddb9f00b7689de7f93ef73099b"}
Jul 08 16:34:15.445622 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.445601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mc44p" event={"ID":"e967c6f7-5197-4648-8ce2-cffcd788026d","Type":"ContainerStarted","Data":"7f6978052ffaf6d4e1a698bf77a316993f1df159b62e1e497be352f57c382fe2"}
Jul 08 16:34:15.457486 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.457462    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"]
Jul 08 16:34:15.459414 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.459370    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vctc7" podStartSLOduration=34.342222949 podStartE2EDuration="40.459354533s" totalImagesPullingTime="6.117131584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:15.458608438 +0000 UTC m=+76.020585823" watchObservedRunningTime="2026-07-08 16:34:15.459354533 +0000 UTC m=+76.021331917"
Jul 08 16:34:15.461102 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:15.461079    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ded0201_533f_4b2b_b270_9605778b186c.slice/crio-f749a05edaf39240d088dd4c2d698807da2b6e976a622084f2f252b39caceec1 WatchSource:0}: Error finding container f749a05edaf39240d088dd4c2d698807da2b6e976a622084f2f252b39caceec1: Status 404 returned error can't find the container with id f749a05edaf39240d088dd4c2d698807da2b6e976a622084f2f252b39caceec1
Jul 08 16:34:15.474586 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.474550    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xgl2r" podStartSLOduration=34.16916269 podStartE2EDuration="39.474538312s" totalImagesPullingTime="5.305375622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:15.474522277 +0000 UTC m=+76.036499662" watchObservedRunningTime="2026-07-08 16:34:15.474538312 +0000 UTC m=+76.036515695"
Jul 08 16:34:15.489312 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:15.489246    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mc44p" podStartSLOduration=67.492280201 podStartE2EDuration="1m14.489233688s" totalImagesPullingTime="6.996953487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:15.489013399 +0000 UTC m=+76.050990813" watchObservedRunningTime="2026-07-08 16:34:15.489233688 +0000 UTC m=+76.051211113"
Jul 08 16:34:16.451437 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:16.451402    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck" event={"ID":"6ded0201-533f-4b2b-b270-9605778b186c","Type":"ContainerStarted","Data":"f749a05edaf39240d088dd4c2d698807da2b6e976a622084f2f252b39caceec1"}
Jul 08 16:34:17.456884 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:17.456836    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck" event={"ID":"6ded0201-533f-4b2b-b270-9605778b186c","Type":"ContainerStarted","Data":"35c713352b3385bf0ce6cf4949bcbd61cffd7c77166b1323f427be9db1489f73"}
Jul 08 16:34:17.457316 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:17.457013    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:17.461675 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:17.461652    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck"
Jul 08 16:34:17.482268 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:17.482220    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jf9ck" podStartSLOduration=1.918775127 podStartE2EDuration="3.482207016s" totalImagesPullingTime="1.563431889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:17.480301963 +0000 UTC m=+78.042279348" watchObservedRunningTime="2026-07-08 16:34:17.482207016 +0000 UTC m=+78.044184407"
Jul 08 16:34:21.023403 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:21.023373    2562 scope.go:117] "RemoveContainer" containerID="2f4f428e88b26219aa6430ee6ea73d1d02f71250685fa6fd2ebf505e6d999053"
Jul 08 16:34:21.023785 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:34:21.023520    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-tnz7d_openshift-console-operator(84b7f28c-4a45-41ad-af20-44ca3821d67f)\"" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podUID="84b7f28c-4a45-41ad-af20-44ca3821d67f"
Jul 08 16:34:25.454467 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:25.454340    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vctc7"
Jul 08 16:34:27.445239 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.445192    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-68h7g"]
Jul 08 16:34:27.449121 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.449095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.452313 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.452288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 16:34:27.452453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.452288    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 16:34:27.452588 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.452565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-klr5s\""
Jul 08 16:34:27.453358 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.453339    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 16:34:27.453358 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.453350    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 16:34:27.532497 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-sys\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532697 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbca7f1b-2c6d-4985-848d-585fb86a554a-metrics-client-ca\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532697 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532525    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-root\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532697 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-wtmp\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532697 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-textfile\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532697 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhmb\" (UniqueName: \"kubernetes.io/projected/cbca7f1b-2c6d-4985-848d-585fb86a554a-kube-api-access-5qhmb\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532963 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532963 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-accelerators-collector-config\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.532963 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.532862    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-tls\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633617 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633579    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-tls\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633784 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-sys\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633784 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbca7f1b-2c6d-4985-848d-585fb86a554a-metrics-client-ca\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633784 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-root\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633940 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633801    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-sys\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633940 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-wtmp\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633940 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633806    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-root\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633940 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633882    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-textfile\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.633940 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhmb\" (UniqueName: \"kubernetes.io/projected/cbca7f1b-2c6d-4985-848d-585fb86a554a-kube-api-access-5qhmb\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.634160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633969    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-wtmp\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.634160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.633976    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.634160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.634008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-accelerators-collector-config\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.634442 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.634415    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbca7f1b-2c6d-4985-848d-585fb86a554a-metrics-client-ca\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.634573 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.634457    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-textfile\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.634573 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.634546    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-accelerators-collector-config\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.636294 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.636272    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.636441 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.636421    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbca7f1b-2c6d-4985-848d-585fb86a554a-node-exporter-tls\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.642376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.642350    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhmb\" (UniqueName: \"kubernetes.io/projected/cbca7f1b-2c6d-4985-848d-585fb86a554a-kube-api-access-5qhmb\") pod \"node-exporter-68h7g\" (UID: \"cbca7f1b-2c6d-4985-848d-585fb86a554a\") " pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.762002 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.761907    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68h7g"
Jul 08 16:34:27.772425 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:34:27.772390    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbca7f1b_2c6d_4985_848d_585fb86a554a.slice/crio-a996b2bb28b2cee77714463a698015145117ef3b68cfb727e3c0b605c4553b6b WatchSource:0}: Error finding container a996b2bb28b2cee77714463a698015145117ef3b68cfb727e3c0b605c4553b6b: Status 404 returned error can't find the container with id a996b2bb28b2cee77714463a698015145117ef3b68cfb727e3c0b605c4553b6b
Jul 08 16:34:27.839262 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.839222    2562 patch_prober.go:28] interesting pod/image-registry-6db7d84484-znjz9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 16:34:27.839422 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:27.839279    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" podUID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:34:28.502406 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:28.502365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68h7g" event={"ID":"cbca7f1b-2c6d-4985-848d-585fb86a554a","Type":"ContainerStarted","Data":"a996b2bb28b2cee77714463a698015145117ef3b68cfb727e3c0b605c4553b6b"}
Jul 08 16:34:29.507292 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:29.507256    2562 generic.go:356] "Generic (PLEG): container finished" podID="cbca7f1b-2c6d-4985-848d-585fb86a554a" containerID="0ded03783211b36db57bac3359b383107de3b60563405dd551d46f503f20b5c1" exitCode=0
Jul 08 16:34:29.507669 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:29.507314    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68h7g" event={"ID":"cbca7f1b-2c6d-4985-848d-585fb86a554a","Type":"ContainerDied","Data":"0ded03783211b36db57bac3359b383107de3b60563405dd551d46f503f20b5c1"}
Jul 08 16:34:30.415946 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:30.415916    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:34:30.514108 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:30.514074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68h7g" event={"ID":"cbca7f1b-2c6d-4985-848d-585fb86a554a","Type":"ContainerStarted","Data":"6ab0c40a62b9941b8880bae6f9b69a806d18efa8399aca12a22779d70b588c59"}
Jul 08 16:34:30.514538 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:30.514127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68h7g" event={"ID":"cbca7f1b-2c6d-4985-848d-585fb86a554a","Type":"ContainerStarted","Data":"495949b825658720d8f3d6326448100abb03ebc87e9a65c58dc84654cf179a45"}
Jul 08 16:34:30.535003 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:30.534947    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-68h7g" podStartSLOduration=2.322784909 podStartE2EDuration="3.5349255s" totalImagesPullingTime="1.14826826s" totalInitContainerRuntime="63.872331ms" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:30.53338786 +0000 UTC m=+91.095365244" watchObservedRunningTime="2026-07-08 16:34:30.5349255 +0000 UTC m=+91.096902886"
Jul 08 16:34:35.023356 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:35.023332    2562 scope.go:117] "RemoveContainer" containerID="2f4f428e88b26219aa6430ee6ea73d1d02f71250685fa6fd2ebf505e6d999053"
Jul 08 16:34:35.535782 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:35.535757    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:34:35.535978 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:35.535853    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" event={"ID":"84b7f28c-4a45-41ad-af20-44ca3821d67f","Type":"ContainerStarted","Data":"bb5c8cd770910726ddc6d90a1a0f3f20305d56d6e87fd0a46bcf36e8409d32dc"}
Jul 08 16:34:35.536183 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:35.536158    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:34:35.541070 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:35.541045    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d"
Jul 08 16:34:35.567144 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:35.567097    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-tnz7d" podStartSLOduration=51.902130013 podStartE2EDuration="59.567082065s" totalImagesPullingTime="7.664952052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:35.565188458 +0000 UTC m=+96.127165843" watchObservedRunningTime="2026-07-08 16:34:35.567082065 +0000 UTC m=+96.129059508"
Jul 08 16:34:38.399850 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:38.399797    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-64tq5"
Jul 08 16:34:40.055318 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:40.055279    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6db7d84484-znjz9"]
Jul 08 16:34:51.594907 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:51.594873    2562 generic.go:356] "Generic (PLEG): container finished" podID="db950e2c-5b14-484d-941d-879cff4b3343" containerID="5ddbc877005d1229559e9c76c4b0cb3074f8f3ec61b8f5f70b5600a42639cf09" exitCode=0
Jul 08 16:34:51.595324 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:51.594954    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq" event={"ID":"db950e2c-5b14-484d-941d-879cff4b3343","Type":"ContainerDied","Data":"5ddbc877005d1229559e9c76c4b0cb3074f8f3ec61b8f5f70b5600a42639cf09"}
Jul 08 16:34:51.595372 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:51.595326    2562 scope.go:117] "RemoveContainer" containerID="5ddbc877005d1229559e9c76c4b0cb3074f8f3ec61b8f5f70b5600a42639cf09"
Jul 08 16:34:52.600659 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:34:52.600622    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xh8sq" event={"ID":"db950e2c-5b14-484d-941d-879cff4b3343","Type":"ContainerStarted","Data":"a80bd2b3feffe7c148cf7a5f34243481e13f51e56100de0d9491d202ef6cc877"}
Jul 08 16:35:01.637299 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:01.637262    2562 generic.go:356] "Generic (PLEG): container finished" podID="113bd2ab-b61a-41f1-a106-0336460bfae4" containerID="723c4f599f6ccb68ca30f2a7c9b94cddc63c355d691ca949729d20c2cc092001" exitCode=0
Jul 08 16:35:01.637780 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:01.637342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp" event={"ID":"113bd2ab-b61a-41f1-a106-0336460bfae4","Type":"ContainerDied","Data":"723c4f599f6ccb68ca30f2a7c9b94cddc63c355d691ca949729d20c2cc092001"}
Jul 08 16:35:01.637780 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:01.637682    2562 scope.go:117] "RemoveContainer" containerID="723c4f599f6ccb68ca30f2a7c9b94cddc63c355d691ca949729d20c2cc092001"
Jul 08 16:35:01.638904 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:01.638872    2562 generic.go:356] "Generic (PLEG): container finished" podID="12d88f79-f370-45df-9935-944f43c1c1d7" containerID="b990693d71607dead7278a5e502ded6a24bde89be7e87cc262f3bc9a36bc5291" exitCode=0
Jul 08 16:35:01.639017 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:01.638921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7" event={"ID":"12d88f79-f370-45df-9935-944f43c1c1d7","Type":"ContainerDied","Data":"b990693d71607dead7278a5e502ded6a24bde89be7e87cc262f3bc9a36bc5291"}
Jul 08 16:35:01.639195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:01.639179    2562 scope.go:117] "RemoveContainer" containerID="b990693d71607dead7278a5e502ded6a24bde89be7e87cc262f3bc9a36bc5291"
Jul 08 16:35:02.644044 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:02.644013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-67sxp" event={"ID":"113bd2ab-b61a-41f1-a106-0336460bfae4","Type":"ContainerStarted","Data":"16033a2453af946d4e315baada1a8a1be9da2e1c9afb915f5cb4ecb740f85d4d"}
Jul 08 16:35:02.645661 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:02.645639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-fqgl7" event={"ID":"12d88f79-f370-45df-9935-944f43c1c1d7","Type":"ContainerStarted","Data":"d08e562ab9f0aa3336a6334b8215570abafb69447aecea1c6312a0123ec5271c"}
Jul 08 16:35:05.076195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.076141    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" podUID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" containerName="registry" containerID="cri-o://a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021" gracePeriod=30
Jul 08 16:35:05.325356 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.325333    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:35:05.440538 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440504    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-certificates\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.440538 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440543    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-installation-pull-secrets\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.440740 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440570    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.440740 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/894c4f16-253b-4e22-8fc7-1fe5aa158afc-ca-trust-extracted\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.440740 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-bound-sa-token\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.440740 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-image-registry-private-configuration\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.440995 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.440957    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs5fd\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-kube-api-access-vs5fd\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.441142 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.441034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-trusted-ca\") pod \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\" (UID: \"894c4f16-253b-4e22-8fc7-1fe5aa158afc\") "
Jul 08 16:35:05.441223 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.441195    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:35:05.441309 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.441291    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-certificates\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.441722 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.441696    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:35:05.443157 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.443129    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:35:05.443270 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.443242    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:35:05.443339 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.443273    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:35:05.443497 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.443470    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-kube-api-access-vs5fd" (OuterVolumeSpecName: "kube-api-access-vs5fd") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "kube-api-access-vs5fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:35:05.443607 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.443586    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:35:05.449676 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.449654    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894c4f16-253b-4e22-8fc7-1fe5aa158afc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "894c4f16-253b-4e22-8fc7-1fe5aa158afc" (UID: "894c4f16-253b-4e22-8fc7-1fe5aa158afc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:35:05.542621 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542594    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs5fd\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-kube-api-access-vs5fd\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.542621 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542618    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/894c4f16-253b-4e22-8fc7-1fe5aa158afc-trusted-ca\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.542621 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542627    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-installation-pull-secrets\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.542850 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542636    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-registry-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.542850 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542644    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/894c4f16-253b-4e22-8fc7-1fe5aa158afc-ca-trust-extracted\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.542850 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542652    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/894c4f16-253b-4e22-8fc7-1fe5aa158afc-bound-sa-token\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.542850 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.542662    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/894c4f16-253b-4e22-8fc7-1fe5aa158afc-image-registry-private-configuration\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:35:05.658532 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.658497    2562 generic.go:356] "Generic (PLEG): container finished" podID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" containerID="a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021" exitCode=0
Jul 08 16:35:05.658685 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.658541    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db7d84484-znjz9"
Jul 08 16:35:05.658685 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.658568    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" event={"ID":"894c4f16-253b-4e22-8fc7-1fe5aa158afc","Type":"ContainerDied","Data":"a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021"}
Jul 08 16:35:05.658685 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.658630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db7d84484-znjz9" event={"ID":"894c4f16-253b-4e22-8fc7-1fe5aa158afc","Type":"ContainerDied","Data":"67f1ff3f9a0e354db2f7bec8da7730e1cbe75600e41ee67148ccd06f9bc0aba0"}
Jul 08 16:35:05.658685 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.658651    2562 scope.go:117] "RemoveContainer" containerID="a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021"
Jul 08 16:35:05.667149 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.667131    2562 scope.go:117] "RemoveContainer" containerID="a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021"
Jul 08 16:35:05.667391 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:35:05.667375    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021\": container with ID starting with a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021 not found: ID does not exist" containerID="a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021"
Jul 08 16:35:05.667438 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.667398    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021"} err="failed to get container status \"a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021\": rpc error: code = NotFound desc = could not find container \"a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021\": container with ID starting with a2b99115a90e3f5a13dfa10644652e6703e21c6c8fe3b5ad3dc14cff26372021 not found: ID does not exist"
Jul 08 16:35:05.683130 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.683107    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6db7d84484-znjz9"]
Jul 08 16:35:05.689842 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:05.689797    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6db7d84484-znjz9"]
Jul 08 16:35:06.028135 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:35:06.028105    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" path="/var/lib/kubelet/pods/894c4f16-253b-4e22-8fc7-1fe5aa158afc/volumes"
Jul 08 16:36:48.250131 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.250096    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"]
Jul 08 16:36:48.250604 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.250376    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" containerName="registry"
Jul 08 16:36:48.250604 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.250387    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" containerName="registry"
Jul 08 16:36:48.250604 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.250447    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="894c4f16-253b-4e22-8fc7-1fe5aa158afc" containerName="registry"
Jul 08 16:36:48.253628 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.253611    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.256847 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.256810    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 08 16:36:48.256992 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.256832    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 08 16:36:48.256992 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.256876    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 08 16:36:48.256992 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.256882    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 08 16:36:48.256992 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.256873    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 08 16:36:48.257958 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.257941    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 08 16:36:48.258002 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.257971    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 08 16:36:48.264250 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.264230    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"]
Jul 08 16:36:48.402685 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.402640    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.402685 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.402684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.402954 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.402707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-ca\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.402954 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.402798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wllng\" (UniqueName: \"kubernetes.io/projected/68db831d-80a1-45b1-b6dd-d658ba506378-kube-api-access-wllng\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.402954 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.402849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-hub\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.402954 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.402878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/68db831d-80a1-45b1-b6dd-d658ba506378-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.503858 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.503744    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.503858 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.503783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.503858 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.503811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-ca\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.504135 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.503878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wllng\" (UniqueName: \"kubernetes.io/projected/68db831d-80a1-45b1-b6dd-d658ba506378-kube-api-access-wllng\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.504135 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.503898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-hub\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.504135 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.503934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/68db831d-80a1-45b1-b6dd-d658ba506378-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.504727 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.504693    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/68db831d-80a1-45b1-b6dd-d658ba506378-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.506411 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.506383    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-ca\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.506536 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.506436    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.506536 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.506449    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-hub\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.506536 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.506476    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68db831d-80a1-45b1-b6dd-d658ba506378-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.512760 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.512734    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wllng\" (UniqueName: \"kubernetes.io/projected/68db831d-80a1-45b1-b6dd-d658ba506378-kube-api-access-wllng\") pod \"cluster-proxy-proxy-agent-7c97dc4c7b-2w82t\" (UID: \"68db831d-80a1-45b1-b6dd-d658ba506378\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.572754 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.572700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"
Jul 08 16:36:48.707692 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:48.707657    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t"]
Jul 08 16:36:48.710759 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:36:48.710723    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68db831d_80a1_45b1_b6dd_d658ba506378.slice/crio-174f8b192a8e1f5e9dd4e198ea6749dcd57caaa7816f549f23c0da1151978aaf WatchSource:0}: Error finding container 174f8b192a8e1f5e9dd4e198ea6749dcd57caaa7816f549f23c0da1151978aaf: Status 404 returned error can't find the container with id 174f8b192a8e1f5e9dd4e198ea6749dcd57caaa7816f549f23c0da1151978aaf
Jul 08 16:36:49.040212 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:49.040177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t" event={"ID":"68db831d-80a1-45b1-b6dd-d658ba506378","Type":"ContainerStarted","Data":"174f8b192a8e1f5e9dd4e198ea6749dcd57caaa7816f549f23c0da1151978aaf"}
Jul 08 16:36:55.066373 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:55.066332    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t" event={"ID":"68db831d-80a1-45b1-b6dd-d658ba506378","Type":"ContainerStarted","Data":"616c8ec75fbc75a3d755a433c6146c0f8116b919fa4a602c04cb8aa3c21edb64"}
Jul 08 16:36:55.066774 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:55.066384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t" event={"ID":"68db831d-80a1-45b1-b6dd-d658ba506378","Type":"ContainerStarted","Data":"6b061bd8f45ea868f7b70c3b3d125e7cff5db5c1ca376e8f3196f53ab981bec5"}
Jul 08 16:36:55.066774 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:36:55.066404    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t" event={"ID":"68db831d-80a1-45b1-b6dd-d658ba506378","Type":"ContainerStarted","Data":"bd37d94a64c1d9f6cca6093e24d3ef523c4c3bed94ccc33151659016b7615abb"}
Jul 08 16:37:59.884452 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:37:59.884425    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:37:59.889766 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:37:59.889742    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:37:59.895973 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:37:59.895946    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 16:42:49.275089 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.275001    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c97dc4c7b-2w82t" podStartSLOduration=355.868995502 podStartE2EDuration="6m1.274977011s" totalImagesPullingTime="5.405981509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:36:55.085658982 +0000 UTC m=+235.647636366" watchObservedRunningTime="2026-07-08 16:42:49.274977011 +0000 UTC m=+589.836954396"
Jul 08 16:42:49.275863 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.275845    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"]
Jul 08 16:42:49.279464 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.279447    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.282465 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.282439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 16:42:49.282465 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.282454    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 16:42:49.283578 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.283558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-r4bjx\""
Jul 08 16:42:49.283696 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.283585    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ed4fa-predictor-serving-cert\""
Jul 08 16:42:49.283696 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.283632    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ed4fa-kube-rbac-proxy-sar-config\""
Jul 08 16:42:49.289926 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.289905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"]
Jul 08 16:42:49.462640 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.462586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.462808 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.462707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpwl4\" (UniqueName: \"kubernetes.io/projected/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-api-access-fpwl4\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.462808 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.462751    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec8f670-0437-4ba1-89e6-9647e0f54703-proxy-tls\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.563525 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.563425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpwl4\" (UniqueName: \"kubernetes.io/projected/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-api-access-fpwl4\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.563525 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.563481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec8f670-0437-4ba1-89e6-9647e0f54703-proxy-tls\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.563757 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.563541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.564495 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.564465    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.566318 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.566293    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec8f670-0437-4ba1-89e6-9647e0f54703-proxy-tls\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.576511 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.576490    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpwl4\" (UniqueName: \"kubernetes.io/projected/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-api-access-fpwl4\") pod \"success-200-isvc-ed4fa-predictor-774674684-75tb6\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") " pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.591225 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.591189    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:42:49.765000 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.764975    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"]
Jul 08 16:42:49.767899 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:42:49.767874    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec8f670_0437_4ba1_89e6_9647e0f54703.slice/crio-498e98748757dd64fe44c6458b7303c99cb18d3d2f02cbefe0326d0eb9a6671e WatchSource:0}: Error finding container 498e98748757dd64fe44c6458b7303c99cb18d3d2f02cbefe0326d0eb9a6671e: Status 404 returned error can't find the container with id 498e98748757dd64fe44c6458b7303c99cb18d3d2f02cbefe0326d0eb9a6671e
Jul 08 16:42:49.769880 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:49.769861    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 16:42:50.390400 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:42:50.390358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" event={"ID":"8ec8f670-0437-4ba1-89e6-9647e0f54703","Type":"ContainerStarted","Data":"498e98748757dd64fe44c6458b7303c99cb18d3d2f02cbefe0326d0eb9a6671e"}
Jul 08 16:43:03.256216 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:03.256184    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:43:03.256637 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:03.256402    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:43:03.459609 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:03.459573    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" event={"ID":"8ec8f670-0437-4ba1-89e6-9647e0f54703","Type":"ContainerStarted","Data":"2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5"}
Jul 08 16:43:06.474587 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:06.474541    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" event={"ID":"8ec8f670-0437-4ba1-89e6-9647e0f54703","Type":"ContainerStarted","Data":"602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d"}
Jul 08 16:43:06.475122 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:06.474623    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:43:06.475122 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:06.474646    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:43:06.476059 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:06.476034    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Jul 08 16:43:06.493943 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:06.493809    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podStartSLOduration=1.763174699 podStartE2EDuration="17.493791026s" totalImagesPullingTime="15.730616327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:42:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:43:06.493671656 +0000 UTC m=+607.055649040" watchObservedRunningTime="2026-07-08 16:43:06.493791026 +0000 UTC m=+607.055768411"
Jul 08 16:43:07.478876 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:07.478811    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Jul 08 16:43:12.482743 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:12.482715    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:43:12.483176 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:12.483141    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Jul 08 16:43:22.484070 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:22.483981    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Jul 08 16:43:32.483871 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:32.483815    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Jul 08 16:43:42.483256 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:42.483213    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Jul 08 16:43:52.483805 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:43:52.483774    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:44:29.022042 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.022007    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"]
Jul 08 16:44:29.024265 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.024247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.026700 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.026675    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-ed4fa-serving-cert\""
Jul 08 16:44:29.026864 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.026700    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-ed4fa-kube-rbac-proxy-sar-config\""
Jul 08 16:44:29.033788 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.033765    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"]
Jul 08 16:44:29.162227 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.162183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be8a854-ec39-4f0d-bf33-067e97ba024d-proxy-tls\") pod \"switch-graph-ed4fa-6678548b58-ft22c\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") " pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.162431 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.162270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be8a854-ec39-4f0d-bf33-067e97ba024d-openshift-service-ca-bundle\") pod \"switch-graph-ed4fa-6678548b58-ft22c\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") " pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.263453 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.263421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be8a854-ec39-4f0d-bf33-067e97ba024d-openshift-service-ca-bundle\") pod \"switch-graph-ed4fa-6678548b58-ft22c\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") " pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.263606 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.263478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be8a854-ec39-4f0d-bf33-067e97ba024d-proxy-tls\") pod \"switch-graph-ed4fa-6678548b58-ft22c\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") " pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.264073 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.264055    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be8a854-ec39-4f0d-bf33-067e97ba024d-openshift-service-ca-bundle\") pod \"switch-graph-ed4fa-6678548b58-ft22c\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") " pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.265933 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.265911    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be8a854-ec39-4f0d-bf33-067e97ba024d-proxy-tls\") pod \"switch-graph-ed4fa-6678548b58-ft22c\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") " pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.335078 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.335048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:29.460300 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.460173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"]
Jul 08 16:44:29.463040 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:44:29.463014    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be8a854_ec39_4f0d_bf33_067e97ba024d.slice/crio-ea942dabfb2d8e6300ab1c3965541574b064bd09791dd9384c80845460e0120e WatchSource:0}: Error finding container ea942dabfb2d8e6300ab1c3965541574b064bd09791dd9384c80845460e0120e: Status 404 returned error can't find the container with id ea942dabfb2d8e6300ab1c3965541574b064bd09791dd9384c80845460e0120e
Jul 08 16:44:29.798585 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:29.798499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" event={"ID":"1be8a854-ec39-4f0d-bf33-067e97ba024d","Type":"ContainerStarted","Data":"ea942dabfb2d8e6300ab1c3965541574b064bd09791dd9384c80845460e0120e"}
Jul 08 16:44:32.813256 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:32.813222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" event={"ID":"1be8a854-ec39-4f0d-bf33-067e97ba024d","Type":"ContainerStarted","Data":"05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db"}
Jul 08 16:44:32.813633 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:32.813301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:32.831977 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:32.831934    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podStartSLOduration=1.368519198 podStartE2EDuration="3.831921035s" totalImagesPullingTime="2.463401837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:32.831384669 +0000 UTC m=+693.393362066" watchObservedRunningTime="2026-07-08 16:44:32.831921035 +0000 UTC m=+693.393898418"
Jul 08 16:44:38.822996 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:38.822969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:39.210924 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.210897    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"]
Jul 08 16:44:39.211094 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.211072    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" containerID="cri-o://05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db" gracePeriod=30
Jul 08 16:44:39.367044 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.367013    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"]
Jul 08 16:44:39.367274 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.367231    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" containerID="cri-o://2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5" gracePeriod=30
Jul 08 16:44:39.367352 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.367282    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kube-rbac-proxy" containerID="cri-o://602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d" gracePeriod=30
Jul 08 16:44:39.427887 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.427862    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"]
Jul 08 16:44:39.430205 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.430189    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.432753 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.432735    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8ceba-predictor-serving-cert\""
Jul 08 16:44:39.432879 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.432766    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8ceba-kube-rbac-proxy-sar-config\""
Jul 08 16:44:39.441720 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.441702    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"]
Jul 08 16:44:39.551446 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.551416    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.551614 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.551453    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-proxy-tls\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.551614 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.551557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzlh\" (UniqueName: \"kubernetes.io/projected/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-api-access-vkzlh\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.651957 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.651917    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.651957 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.651959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-proxy-tls\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.652164 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.652002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzlh\" (UniqueName: \"kubernetes.io/projected/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-api-access-vkzlh\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.652665 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.652646    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.654460 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.654438    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-proxy-tls\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.663518 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.663496    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzlh\" (UniqueName: \"kubernetes.io/projected/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-api-access-vkzlh\") pod \"success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") " pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.743198 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.743131    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:39.844232 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.844199    2562 generic.go:356] "Generic (PLEG): container finished" podID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerID="602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d" exitCode=2
Jul 08 16:44:39.844677 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.844274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" event={"ID":"8ec8f670-0437-4ba1-89e6-9647e0f54703","Type":"ContainerDied","Data":"602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d"}
Jul 08 16:44:39.874501 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:39.874390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"]
Jul 08 16:44:39.876465 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:44:39.876444    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5794e7e_1a94_4ccd_b00d_8d9adbea100a.slice/crio-05081fd59da50c76c501e2b2a79ba7681eda8010b521ac2b345d3600d75bf846 WatchSource:0}: Error finding container 05081fd59da50c76c501e2b2a79ba7681eda8010b521ac2b345d3600d75bf846: Status 404 returned error can't find the container with id 05081fd59da50c76c501e2b2a79ba7681eda8010b521ac2b345d3600d75bf846
Jul 08 16:44:40.849786 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.849753    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" event={"ID":"d5794e7e-1a94-4ccd-b00d-8d9adbea100a","Type":"ContainerStarted","Data":"65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e"}
Jul 08 16:44:40.850175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.849816    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:40.850175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.849859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" event={"ID":"d5794e7e-1a94-4ccd-b00d-8d9adbea100a","Type":"ContainerStarted","Data":"8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305"}
Jul 08 16:44:40.850175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.849908    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:40.850175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.849932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" event={"ID":"d5794e7e-1a94-4ccd-b00d-8d9adbea100a","Type":"ContainerStarted","Data":"05081fd59da50c76c501e2b2a79ba7681eda8010b521ac2b345d3600d75bf846"}
Jul 08 16:44:40.850794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.850770    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:44:40.868619 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:40.868573    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podStartSLOduration=1.8685589820000001 podStartE2EDuration="1.868558982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:40.866812948 +0000 UTC m=+701.428790329" watchObservedRunningTime="2026-07-08 16:44:40.868558982 +0000 UTC m=+701.430536365"
Jul 08 16:44:41.854022 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:41.853986    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:44:42.512761 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.512739    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:44:42.677374 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.677282    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-rbac-proxy-sar-config\") pod \"8ec8f670-0437-4ba1-89e6-9647e0f54703\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") "
Jul 08 16:44:42.677374 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.677326    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec8f670-0437-4ba1-89e6-9647e0f54703-proxy-tls\") pod \"8ec8f670-0437-4ba1-89e6-9647e0f54703\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") "
Jul 08 16:44:42.677594 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.677446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpwl4\" (UniqueName: \"kubernetes.io/projected/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-api-access-fpwl4\") pod \"8ec8f670-0437-4ba1-89e6-9647e0f54703\" (UID: \"8ec8f670-0437-4ba1-89e6-9647e0f54703\") "
Jul 08 16:44:42.677741 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.677716    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8ec8f670-0437-4ba1-89e6-9647e0f54703" (UID: "8ec8f670-0437-4ba1-89e6-9647e0f54703"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:44:42.679588 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.679565    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-api-access-fpwl4" (OuterVolumeSpecName: "kube-api-access-fpwl4") pod "8ec8f670-0437-4ba1-89e6-9647e0f54703" (UID: "8ec8f670-0437-4ba1-89e6-9647e0f54703"). InnerVolumeSpecName "kube-api-access-fpwl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:44:42.679654 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.679604    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ec8f670-0437-4ba1-89e6-9647e0f54703-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8ec8f670-0437-4ba1-89e6-9647e0f54703" (UID: "8ec8f670-0437-4ba1-89e6-9647e0f54703"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:44:42.778909 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.778859    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpwl4\" (UniqueName: \"kubernetes.io/projected/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-api-access-fpwl4\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:44:42.778909 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.778903    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ec8f670-0437-4ba1-89e6-9647e0f54703-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:44:42.778909 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.778915    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec8f670-0437-4ba1-89e6-9647e0f54703-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:44:42.858795 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.858762    2562 generic.go:356] "Generic (PLEG): container finished" podID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerID="2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5" exitCode=0
Jul 08 16:44:42.859235 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.858808    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"
Jul 08 16:44:42.859235 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.858848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" event={"ID":"8ec8f670-0437-4ba1-89e6-9647e0f54703","Type":"ContainerDied","Data":"2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5"}
Jul 08 16:44:42.859235 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.858909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" event={"ID":"8ec8f670-0437-4ba1-89e6-9647e0f54703","Type":"ContainerDied","Data":"498e98748757dd64fe44c6458b7303c99cb18d3d2f02cbefe0326d0eb9a6671e"}
Jul 08 16:44:42.859235 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.858935    2562 scope.go:117] "RemoveContainer" containerID="602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d"
Jul 08 16:44:42.867165 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.867149    2562 scope.go:117] "RemoveContainer" containerID="2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5"
Jul 08 16:44:42.874254 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.874236    2562 scope.go:117] "RemoveContainer" containerID="602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d"
Jul 08 16:44:42.874492 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:44:42.874471    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d\": container with ID starting with 602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d not found: ID does not exist" containerID="602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d"
Jul 08 16:44:42.874554 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.874500    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d"} err="failed to get container status \"602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d\": rpc error: code = NotFound desc = could not find container \"602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d\": container with ID starting with 602a789cb82f1e148f7670f0c87f1ea2755864294934568ac865b513cdf4af7d not found: ID does not exist"
Jul 08 16:44:42.874554 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.874517    2562 scope.go:117] "RemoveContainer" containerID="2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5"
Jul 08 16:44:42.874760 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:44:42.874743    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5\": container with ID starting with 2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5 not found: ID does not exist" containerID="2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5"
Jul 08 16:44:42.874802 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.874766    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5"} err="failed to get container status \"2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5\": rpc error: code = NotFound desc = could not find container \"2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5\": container with ID starting with 2967a74c2978fcdd78de0011e343b0fc9431766b6d82e72fba84144e8b0ce4b5 not found: ID does not exist"
Jul 08 16:44:42.880021 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.880003    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"]
Jul 08 16:44:42.884618 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:42.884600    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6"]
Jul 08 16:44:43.479472 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:43.479429    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.24:8643/healthz\": context deadline exceeded"
Jul 08 16:44:43.483665 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:43.483645    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ed4fa-predictor-774674684-75tb6" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: i/o timeout"
Jul 08 16:44:43.821684 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:43.821597    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:44:44.028071 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:44.028040    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" path="/var/lib/kubelet/pods/8ec8f670-0437-4ba1-89e6-9647e0f54703/volumes"
Jul 08 16:44:46.858984 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:46.858953    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:44:46.859562 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:46.859533    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:44:48.822402 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:48.822359    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:44:53.821614 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:53.821519    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:44:53.821972 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:53.821673    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:44:56.860491 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:56.860456    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:44:58.822283 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:44:58.822249    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:03.822028 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:03.821986    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:06.859958 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:06.859923    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:45:08.822085 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:08.822043    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:09.037393 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037354    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"]
Jul 08 16:45:09.037725 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037703    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container"
Jul 08 16:45:09.037725 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037727    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container"
Jul 08 16:45:09.037899 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037752    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kube-rbac-proxy"
Jul 08 16:45:09.037899 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037758    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kube-rbac-proxy"
Jul 08 16:45:09.037899 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037853    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kserve-container"
Jul 08 16:45:09.037899 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.037866    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec8f670-0437-4ba1-89e6-9647e0f54703" containerName="kube-rbac-proxy"
Jul 08 16:45:09.042397 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.042381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.044548 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.044530    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Jul 08 16:45:09.044640 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.044536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Jul 08 16:45:09.052076 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.052053    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"]
Jul 08 16:45:09.079017 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.078961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.079017 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.079009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3a74be-312a-403c-8f10-89053f52c915-openshift-service-ca-bundle\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.180045 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.180017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.180220 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.180063    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3a74be-312a-403c-8f10-89053f52c915-openshift-service-ca-bundle\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.180220 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:45:09.180156    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found
Jul 08 16:45:09.180353 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:45:09.180228    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls podName:3f3a74be-312a-403c-8f10-89053f52c915 nodeName:}" failed. No retries permitted until 2026-07-08 16:45:09.680207308 +0000 UTC m=+730.242184674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls") pod "model-chainer-bdf5fb5f7-bnnvv" (UID: "3f3a74be-312a-403c-8f10-89053f52c915") : secret "model-chainer-serving-cert" not found
Jul 08 16:45:09.180845 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.180809    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3a74be-312a-403c-8f10-89053f52c915-openshift-service-ca-bundle\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.357567 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.357545    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:45:09.381444 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.381421    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be8a854-ec39-4f0d-bf33-067e97ba024d-proxy-tls\") pod \"1be8a854-ec39-4f0d-bf33-067e97ba024d\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") "
Jul 08 16:45:09.381573 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.381471    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be8a854-ec39-4f0d-bf33-067e97ba024d-openshift-service-ca-bundle\") pod \"1be8a854-ec39-4f0d-bf33-067e97ba024d\" (UID: \"1be8a854-ec39-4f0d-bf33-067e97ba024d\") "
Jul 08 16:45:09.381757 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.381737    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be8a854-ec39-4f0d-bf33-067e97ba024d-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1be8a854-ec39-4f0d-bf33-067e97ba024d" (UID: "1be8a854-ec39-4f0d-bf33-067e97ba024d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:45:09.383492 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.383472    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1be8a854-ec39-4f0d-bf33-067e97ba024d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1be8a854-ec39-4f0d-bf33-067e97ba024d" (UID: "1be8a854-ec39-4f0d-bf33-067e97ba024d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:45:09.482906 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.482871    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be8a854-ec39-4f0d-bf33-067e97ba024d-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:45:09.482906 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.482898    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be8a854-ec39-4f0d-bf33-067e97ba024d-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:45:09.684639 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.684553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.686876 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.686849    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls\") pod \"model-chainer-bdf5fb5f7-bnnvv\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") " pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.954762 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.954680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:09.975931 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.975896    2562 generic.go:356] "Generic (PLEG): container finished" podID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerID="05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db" exitCode=0
Jul 08 16:45:09.976049 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.975952    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"
Jul 08 16:45:09.976049 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.975974    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" event={"ID":"1be8a854-ec39-4f0d-bf33-067e97ba024d","Type":"ContainerDied","Data":"05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db"}
Jul 08 16:45:09.976049 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.976031    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c" event={"ID":"1be8a854-ec39-4f0d-bf33-067e97ba024d","Type":"ContainerDied","Data":"ea942dabfb2d8e6300ab1c3965541574b064bd09791dd9384c80845460e0120e"}
Jul 08 16:45:09.976151 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.976053    2562 scope.go:117] "RemoveContainer" containerID="05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db"
Jul 08 16:45:09.984385 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.984367    2562 scope.go:117] "RemoveContainer" containerID="05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db"
Jul 08 16:45:09.984655 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:45:09.984637    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db\": container with ID starting with 05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db not found: ID does not exist" containerID="05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db"
Jul 08 16:45:09.984708 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.984663    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db"} err="failed to get container status \"05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db\": rpc error: code = NotFound desc = could not find container \"05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db\": container with ID starting with 05af52a5cdb2d301311a23d30fd8fef0e00f331b03de59b0a7d595c8559040db not found: ID does not exist"
Jul 08 16:45:09.998579 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:09.998557    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"]
Jul 08 16:45:10.001385 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.001339    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ed4fa-6678548b58-ft22c"]
Jul 08 16:45:10.029143 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.029109    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" path="/var/lib/kubelet/pods/1be8a854-ec39-4f0d-bf33-067e97ba024d/volumes"
Jul 08 16:45:10.085109 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.085087    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"]
Jul 08 16:45:10.087635 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:45:10.087609    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3a74be_312a_403c_8f10_89053f52c915.slice/crio-451445bd532ccc73aea9d7b24e1e86737abe7829aca1de020ec3edd94b6548a6 WatchSource:0}: Error finding container 451445bd532ccc73aea9d7b24e1e86737abe7829aca1de020ec3edd94b6548a6: Status 404 returned error can't find the container with id 451445bd532ccc73aea9d7b24e1e86737abe7829aca1de020ec3edd94b6548a6
Jul 08 16:45:10.981437 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.981408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" event={"ID":"3f3a74be-312a-403c-8f10-89053f52c915","Type":"ContainerStarted","Data":"574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec"}
Jul 08 16:45:10.981939 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.981453    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" event={"ID":"3f3a74be-312a-403c-8f10-89053f52c915","Type":"ContainerStarted","Data":"451445bd532ccc73aea9d7b24e1e86737abe7829aca1de020ec3edd94b6548a6"}
Jul 08 16:45:10.981939 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.981522    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:10.999157 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:10.999112    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podStartSLOduration=1.999098335 podStartE2EDuration="1.999098335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:45:10.997778882 +0000 UTC m=+731.559756260" watchObservedRunningTime="2026-07-08 16:45:10.999098335 +0000 UTC m=+731.561075718"
Jul 08 16:45:16.859650 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:16.859609    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:45:16.994157 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:16.994124    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:19.158555 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.158517    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"]
Jul 08 16:45:19.158952 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.158763    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" containerID="cri-o://574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec" gracePeriod=30
Jul 08 16:45:19.382088 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.382037    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"]
Jul 08 16:45:19.382441 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.382423    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa"
Jul 08 16:45:19.382523 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.382444    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa"
Jul 08 16:45:19.382578 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.382536    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be8a854-ec39-4f0d-bf33-067e97ba024d" containerName="switch-graph-ed4fa"
Jul 08 16:45:19.384814 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.384796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.387576 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.387558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-fabf2-predictor-serving-cert\""
Jul 08 16:45:19.387663 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.387609    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-fabf2-kube-rbac-proxy-sar-config\""
Jul 08 16:45:19.404052 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.404029    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"]
Jul 08 16:45:19.462752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.462670    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-642nz\" (UniqueName: \"kubernetes.io/projected/75e58397-959f-43bb-9709-4f1b09463274-kube-api-access-642nz\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.462752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.462711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/75e58397-959f-43bb-9709-4f1b09463274-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.462752 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.462745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/75e58397-959f-43bb-9709-4f1b09463274-proxy-tls\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.564148 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.564118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-642nz\" (UniqueName: \"kubernetes.io/projected/75e58397-959f-43bb-9709-4f1b09463274-kube-api-access-642nz\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.564351 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.564155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/75e58397-959f-43bb-9709-4f1b09463274-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.564351 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.564180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/75e58397-959f-43bb-9709-4f1b09463274-proxy-tls\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.564899 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.564878    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/75e58397-959f-43bb-9709-4f1b09463274-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.566705 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.566686    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/75e58397-959f-43bb-9709-4f1b09463274-proxy-tls\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.572642 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.572617    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-642nz\" (UniqueName: \"kubernetes.io/projected/75e58397-959f-43bb-9709-4f1b09463274-kube-api-access-642nz\") pod \"success-200-isvc-fabf2-predictor-7998766446-tv7kw\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") " pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.698741 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.698712    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:19.825005 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:19.824984    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"]
Jul 08 16:45:19.827062 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:45:19.827036    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e58397_959f_43bb_9709_4f1b09463274.slice/crio-4cd1889a1e1eefe6b98ae4709509cea7161ad84e7b2a868dfbe901acac0144ff WatchSource:0}: Error finding container 4cd1889a1e1eefe6b98ae4709509cea7161ad84e7b2a868dfbe901acac0144ff: Status 404 returned error can't find the container with id 4cd1889a1e1eefe6b98ae4709509cea7161ad84e7b2a868dfbe901acac0144ff
Jul 08 16:45:20.029285 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:20.029200    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" event={"ID":"75e58397-959f-43bb-9709-4f1b09463274","Type":"ContainerStarted","Data":"4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465"}
Jul 08 16:45:20.029285 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:20.029274    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:20.029472 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:20.029303    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" event={"ID":"75e58397-959f-43bb-9709-4f1b09463274","Type":"ContainerStarted","Data":"b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc"}
Jul 08 16:45:20.029472 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:20.029330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" event={"ID":"75e58397-959f-43bb-9709-4f1b09463274","Type":"ContainerStarted","Data":"4cd1889a1e1eefe6b98ae4709509cea7161ad84e7b2a868dfbe901acac0144ff"}
Jul 08 16:45:20.046475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:20.046424    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podStartSLOduration=1.046405812 podStartE2EDuration="1.046405812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:45:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:45:20.045309157 +0000 UTC m=+740.607286568" watchObservedRunningTime="2026-07-08 16:45:20.046405812 +0000 UTC m=+740.608383195"
Jul 08 16:45:21.032782 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:21.032749    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:21.034347 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:21.034320    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:45:21.991919 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:21.991876    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:22.038130 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:22.038092    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:45:26.860959 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:26.860931    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:45:26.992191 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:26.992157    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:27.042935 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:27.042907    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:45:27.043482 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:27.043457    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:45:31.991733 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:31.991698    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:31.992246 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:31.991801    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:36.991613 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:36.991573    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:37.044241 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:37.044206    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:45:39.484442 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.484409    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"]
Jul 08 16:45:39.487676 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.487659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.490217 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.490196    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8ceba-serving-cert\""
Jul 08 16:45:39.490304 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.490203    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8ceba-kube-rbac-proxy-sar-config\""
Jul 08 16:45:39.495856 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.495834    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"]
Jul 08 16:45:39.627329 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.627292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-openshift-service-ca-bundle\") pod \"switch-graph-8ceba-5774d86bd6-r5cbl\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") " pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.627529 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.627397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-proxy-tls\") pod \"switch-graph-8ceba-5774d86bd6-r5cbl\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") " pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.728634 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.728598    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-proxy-tls\") pod \"switch-graph-8ceba-5774d86bd6-r5cbl\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") " pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.728816 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.728655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-openshift-service-ca-bundle\") pod \"switch-graph-8ceba-5774d86bd6-r5cbl\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") " pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.729372 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.729347    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-openshift-service-ca-bundle\") pod \"switch-graph-8ceba-5774d86bd6-r5cbl\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") " pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.731011 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.730992    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-proxy-tls\") pod \"switch-graph-8ceba-5774d86bd6-r5cbl\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") " pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.798424 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.798337    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:39.929608 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:39.929520    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"]
Jul 08 16:45:39.932578 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:45:39.932551    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfc49f7_1e6d_4d50_b91d_d223d148ac28.slice/crio-e16fa96a7212b260e7330f187383854aca583e8252e342c87b129c57ef9674c4 WatchSource:0}: Error finding container e16fa96a7212b260e7330f187383854aca583e8252e342c87b129c57ef9674c4: Status 404 returned error can't find the container with id e16fa96a7212b260e7330f187383854aca583e8252e342c87b129c57ef9674c4
Jul 08 16:45:40.110254 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:40.110219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" event={"ID":"3cfc49f7-1e6d-4d50-b91d-d223d148ac28","Type":"ContainerStarted","Data":"4352a3216b61fbf4c5ab108699b1401b684f8f1cca04275f3856eb7d02a25027"}
Jul 08 16:45:40.110419 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:40.110272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" event={"ID":"3cfc49f7-1e6d-4d50-b91d-d223d148ac28","Type":"ContainerStarted","Data":"e16fa96a7212b260e7330f187383854aca583e8252e342c87b129c57ef9674c4"}
Jul 08 16:45:40.110419 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:40.110364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:40.125769 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:40.125723    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podStartSLOduration=1.125707729 podStartE2EDuration="1.125707729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:45:40.125392604 +0000 UTC m=+760.687369989" watchObservedRunningTime="2026-07-08 16:45:40.125707729 +0000 UTC m=+760.687685113"
Jul 08 16:45:41.992200 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:41.992156    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:46.120705 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:46.120680    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:45:46.992892 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:46.992807    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:47.044053 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:47.044012    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:45:49.294960 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.294938    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:49.413416 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.413340    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3a74be-312a-403c-8f10-89053f52c915-openshift-service-ca-bundle\") pod \"3f3a74be-312a-403c-8f10-89053f52c915\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") "
Jul 08 16:45:49.413416 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.413382    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls\") pod \"3f3a74be-312a-403c-8f10-89053f52c915\" (UID: \"3f3a74be-312a-403c-8f10-89053f52c915\") "
Jul 08 16:45:49.413690 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.413670    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f3a74be-312a-403c-8f10-89053f52c915-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3f3a74be-312a-403c-8f10-89053f52c915" (UID: "3f3a74be-312a-403c-8f10-89053f52c915"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:45:49.415378 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.415348    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3f3a74be-312a-403c-8f10-89053f52c915" (UID: "3f3a74be-312a-403c-8f10-89053f52c915"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:45:49.514183 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.514150    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3a74be-312a-403c-8f10-89053f52c915-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:45:49.514183 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:49.514177    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f3a74be-312a-403c-8f10-89053f52c915-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:45:50.153951 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.153914    2562 generic.go:356] "Generic (PLEG): container finished" podID="3f3a74be-312a-403c-8f10-89053f52c915" containerID="574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec" exitCode=0
Jul 08 16:45:50.154138 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.153960    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"
Jul 08 16:45:50.154138 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.154003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" event={"ID":"3f3a74be-312a-403c-8f10-89053f52c915","Type":"ContainerDied","Data":"574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec"}
Jul 08 16:45:50.154138 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.154068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv" event={"ID":"3f3a74be-312a-403c-8f10-89053f52c915","Type":"ContainerDied","Data":"451445bd532ccc73aea9d7b24e1e86737abe7829aca1de020ec3edd94b6548a6"}
Jul 08 16:45:50.154138 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.154097    2562 scope.go:117] "RemoveContainer" containerID="574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec"
Jul 08 16:45:50.162152 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.161941    2562 scope.go:117] "RemoveContainer" containerID="574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec"
Jul 08 16:45:50.162232 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:45:50.162213    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec\": container with ID starting with 574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec not found: ID does not exist" containerID="574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec"
Jul 08 16:45:50.162271 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.162241    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec"} err="failed to get container status \"574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec\": rpc error: code = NotFound desc = could not find container \"574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec\": container with ID starting with 574f106b88d8fd19ed40eefef804fa56708b0304e54e699c5b4aad1fb5723aec not found: ID does not exist"
Jul 08 16:45:50.173211 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.173185    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"]
Jul 08 16:45:50.176913 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:50.176889    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-bdf5fb5f7-bnnvv"]
Jul 08 16:45:52.032680 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:52.032645    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f3a74be-312a-403c-8f10-89053f52c915" path="/var/lib/kubelet/pods/3f3a74be-312a-403c-8f10-89053f52c915/volumes"
Jul 08 16:45:57.044179 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:45:57.044139    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:46:07.044729 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:07.044696    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:46:19.377763 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.377726    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"]
Jul 08 16:46:19.378160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.378088    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer"
Jul 08 16:46:19.378160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.378100    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer"
Jul 08 16:46:19.378160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.378153    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3a74be-312a-403c-8f10-89053f52c915" containerName="model-chainer"
Jul 08 16:46:19.382475 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.382459    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:19.385121 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.385100    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fabf2-serving-cert\""
Jul 08 16:46:19.385228 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.385100    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fabf2-kube-rbac-proxy-sar-config\""
Jul 08 16:46:19.391763 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.391739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"]
Jul 08 16:46:19.444368 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.444333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a14cd999-092f-472b-97c7-563be16c2261-openshift-service-ca-bundle\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:19.444530 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.444382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:19.544790 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.544760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a14cd999-092f-472b-97c7-563be16c2261-openshift-service-ca-bundle\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:19.544968 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.544795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:19.544968 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:46:19.544952    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-fabf2-serving-cert: secret "sequence-graph-fabf2-serving-cert" not found
Jul 08 16:46:19.545037 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:46:19.545020    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls podName:a14cd999-092f-472b-97c7-563be16c2261 nodeName:}" failed. No retries permitted until 2026-07-08 16:46:20.0450002 +0000 UTC m=+800.606977578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls") pod "sequence-graph-fabf2-5f6647c4bf-kcmn7" (UID: "a14cd999-092f-472b-97c7-563be16c2261") : secret "sequence-graph-fabf2-serving-cert" not found
Jul 08 16:46:19.545489 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:19.545471    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a14cd999-092f-472b-97c7-563be16c2261-openshift-service-ca-bundle\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:20.048550 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:20.048518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:20.051097 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:20.051070    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls\") pod \"sequence-graph-fabf2-5f6647c4bf-kcmn7\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") " pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:20.293413 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:20.293381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:20.422448 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:20.422422    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"]
Jul 08 16:46:20.424795 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:46:20.424770    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14cd999_092f_472b_97c7_563be16c2261.slice/crio-56d1376826ccb28abe3faf2bd1ae2f54b53fe5567451b74130831f4ea9fc0e23 WatchSource:0}: Error finding container 56d1376826ccb28abe3faf2bd1ae2f54b53fe5567451b74130831f4ea9fc0e23: Status 404 returned error can't find the container with id 56d1376826ccb28abe3faf2bd1ae2f54b53fe5567451b74130831f4ea9fc0e23
Jul 08 16:46:21.282148 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:21.282113    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" event={"ID":"a14cd999-092f-472b-97c7-563be16c2261","Type":"ContainerStarted","Data":"ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae"}
Jul 08 16:46:21.282340 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:21.282167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" event={"ID":"a14cd999-092f-472b-97c7-563be16c2261","Type":"ContainerStarted","Data":"56d1376826ccb28abe3faf2bd1ae2f54b53fe5567451b74130831f4ea9fc0e23"}
Jul 08 16:46:21.282340 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:21.282210    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:46:21.300433 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:21.300389    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podStartSLOduration=2.30037494 podStartE2EDuration="2.30037494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:46:21.298715239 +0000 UTC m=+801.860692624" watchObservedRunningTime="2026-07-08 16:46:21.30037494 +0000 UTC m=+801.862352324"
Jul 08 16:46:27.293230 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:46:27.293133    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:48:03.287389 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:48:03.287360    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:48:03.288935 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:48:03.288914    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:53:03.323621 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:03.323541    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:53:03.324988 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:03.324961    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:53:54.117583 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.117503    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"]
Jul 08 16:53:54.118368 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.117712    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" containerID="cri-o://4352a3216b61fbf4c5ab108699b1401b684f8f1cca04275f3856eb7d02a25027" gracePeriod=30
Jul 08 16:53:54.291664 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.291629    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"]
Jul 08 16:53:54.291890 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.291867    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" containerID="cri-o://8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305" gracePeriod=30
Jul 08 16:53:54.291949 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.291893    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kube-rbac-proxy" containerID="cri-o://65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e" gracePeriod=30
Jul 08 16:53:54.435109 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.435075    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"]
Jul 08 16:53:54.438446 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.438427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.441140 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.441113    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8befa-kube-rbac-proxy-sar-config\""
Jul 08 16:53:54.441293 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.441223    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8befa-predictor-serving-cert\""
Jul 08 16:53:54.451251 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.451224    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"]
Jul 08 16:53:54.606712 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.606667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7rb6\" (UniqueName: \"kubernetes.io/projected/c0601e45-bb2c-453e-9922-303007fe8fba-kube-api-access-z7rb6\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.606943 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.606805    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0601e45-bb2c-453e-9922-303007fe8fba-proxy-tls\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.606943 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.606871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0601e45-bb2c-453e-9922-303007fe8fba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.708386 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.708289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0601e45-bb2c-453e-9922-303007fe8fba-proxy-tls\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.708386 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.708331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0601e45-bb2c-453e-9922-303007fe8fba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.708386 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.708363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7rb6\" (UniqueName: \"kubernetes.io/projected/c0601e45-bb2c-453e-9922-303007fe8fba-kube-api-access-z7rb6\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.709169 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.709145    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0601e45-bb2c-453e-9922-303007fe8fba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.710806 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.710787    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0601e45-bb2c-453e-9922-303007fe8fba-proxy-tls\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.717602 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.717577    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7rb6\" (UniqueName: \"kubernetes.io/projected/c0601e45-bb2c-453e-9922-303007fe8fba-kube-api-access-z7rb6\") pod \"success-200-isvc-8befa-predictor-85866c45b8-tgk8s\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") " pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.749611 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.749575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:54.892882 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:54.892854    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"]
Jul 08 16:53:54.894906 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:53:54.894879    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0601e45_bb2c_453e_9922_303007fe8fba.slice/crio-21f5800423e166211f83aeb0b9d115a7f2b3637eb1064d5c3c24db9f431d4afd WatchSource:0}: Error finding container 21f5800423e166211f83aeb0b9d115a7f2b3637eb1064d5c3c24db9f431d4afd: Status 404 returned error can't find the container with id 21f5800423e166211f83aeb0b9d115a7f2b3637eb1064d5c3c24db9f431d4afd
Jul 08 16:53:55.184534 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.184499    2562 generic.go:356] "Generic (PLEG): container finished" podID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerID="65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e" exitCode=2
Jul 08 16:53:55.184968 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.184567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" event={"ID":"d5794e7e-1a94-4ccd-b00d-8d9adbea100a","Type":"ContainerDied","Data":"65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e"}
Jul 08 16:53:55.186316 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.186296    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" event={"ID":"c0601e45-bb2c-453e-9922-303007fe8fba","Type":"ContainerStarted","Data":"053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e"}
Jul 08 16:53:55.186434 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.186340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" event={"ID":"c0601e45-bb2c-453e-9922-303007fe8fba","Type":"ContainerStarted","Data":"4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef"}
Jul 08 16:53:55.186434 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.186367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" event={"ID":"c0601e45-bb2c-453e-9922-303007fe8fba","Type":"ContainerStarted","Data":"21f5800423e166211f83aeb0b9d115a7f2b3637eb1064d5c3c24db9f431d4afd"}
Jul 08 16:53:55.186541 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.186507    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:55.186686 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.186666    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:53:55.187858 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.187835    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 08 16:53:55.209073 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:55.209028    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podStartSLOduration=1.209014227 podStartE2EDuration="1.209014227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:53:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:53:55.207054902 +0000 UTC m=+1255.769032280" watchObservedRunningTime="2026-07-08 16:53:55.209014227 +0000 UTC m=+1255.770991611"
Jul 08 16:53:56.118281 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:56.118236    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:53:56.190955 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:56.190914    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 08 16:53:56.854764 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:56.854716    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused"
Jul 08 16:53:56.859994 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:56.859962    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 08 16:53:57.647630 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.647605    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:53:57.838115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.838087    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-rbac-proxy-sar-config\") pod \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") "
Jul 08 16:53:57.838260 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.838182    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkzlh\" (UniqueName: \"kubernetes.io/projected/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-api-access-vkzlh\") pod \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") "
Jul 08 16:53:57.838314 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.838263    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-proxy-tls\") pod \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\" (UID: \"d5794e7e-1a94-4ccd-b00d-8d9adbea100a\") "
Jul 08 16:53:57.838509 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.838487    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d5794e7e-1a94-4ccd-b00d-8d9adbea100a" (UID: "d5794e7e-1a94-4ccd-b00d-8d9adbea100a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:53:57.840305 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.840271    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d5794e7e-1a94-4ccd-b00d-8d9adbea100a" (UID: "d5794e7e-1a94-4ccd-b00d-8d9adbea100a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:53:57.840433 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.840411    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-api-access-vkzlh" (OuterVolumeSpecName: "kube-api-access-vkzlh") pod "d5794e7e-1a94-4ccd-b00d-8d9adbea100a" (UID: "d5794e7e-1a94-4ccd-b00d-8d9adbea100a"). InnerVolumeSpecName "kube-api-access-vkzlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:53:57.939252 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.939214    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:53:57.939252 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.939247    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkzlh\" (UniqueName: \"kubernetes.io/projected/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-kube-api-access-vkzlh\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:53:57.939252 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:57.939258    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5794e7e-1a94-4ccd-b00d-8d9adbea100a-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:53:58.200438 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.200355    2562 generic.go:356] "Generic (PLEG): container finished" podID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerID="8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305" exitCode=0
Jul 08 16:53:58.200438 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.200409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"
Jul 08 16:53:58.200604 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.200437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" event={"ID":"d5794e7e-1a94-4ccd-b00d-8d9adbea100a","Type":"ContainerDied","Data":"8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305"}
Jul 08 16:53:58.200604 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.200505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4" event={"ID":"d5794e7e-1a94-4ccd-b00d-8d9adbea100a","Type":"ContainerDied","Data":"05081fd59da50c76c501e2b2a79ba7681eda8010b521ac2b345d3600d75bf846"}
Jul 08 16:53:58.200604 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.200543    2562 scope.go:117] "RemoveContainer" containerID="65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e"
Jul 08 16:53:58.208680 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.208649    2562 scope.go:117] "RemoveContainer" containerID="8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305"
Jul 08 16:53:58.215772 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.215752    2562 scope.go:117] "RemoveContainer" containerID="65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e"
Jul 08 16:53:58.216033 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:53:58.216016    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e\": container with ID starting with 65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e not found: ID does not exist" containerID="65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e"
Jul 08 16:53:58.216087 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.216042    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e"} err="failed to get container status \"65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e\": rpc error: code = NotFound desc = could not find container \"65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e\": container with ID starting with 65ca11b56139d1c1de7931856fb8f4bc7d873a4eeb67a2ec488cad8b65ea917e not found: ID does not exist"
Jul 08 16:53:58.216087 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.216059    2562 scope.go:117] "RemoveContainer" containerID="8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305"
Jul 08 16:53:58.216314 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:53:58.216290    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305\": container with ID starting with 8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305 not found: ID does not exist" containerID="8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305"
Jul 08 16:53:58.216363 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.216323    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305"} err="failed to get container status \"8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305\": rpc error: code = NotFound desc = could not find container \"8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305\": container with ID starting with 8b0ffbd4a1942e8998e046888331bd5ab5e0bf868aaff43fe4f5399067fa6305 not found: ID does not exist"
Jul 08 16:53:58.218626 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.218608    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"]
Jul 08 16:53:58.222422 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:53:58.222404    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8ceba-predictor-5567b9d7d6-cqvc4"]
Jul 08 16:54:00.032035 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:00.030443    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" path="/var/lib/kubelet/pods/d5794e7e-1a94-4ccd-b00d-8d9adbea100a/volumes"
Jul 08 16:54:01.118597 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:01.118560    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:01.195517 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:01.195485    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:54:01.196136 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:01.196097    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 08 16:54:06.119098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:06.119060    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:06.119551 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:06.119173    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:54:11.119255 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:11.119217    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:11.196809 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:11.196774    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 08 16:54:16.118766 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:16.118717    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:21.118324 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:21.118286    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:21.196338 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:21.196301    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 08 16:54:24.314482 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.314445    2562 generic.go:356] "Generic (PLEG): container finished" podID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerID="4352a3216b61fbf4c5ab108699b1401b684f8f1cca04275f3856eb7d02a25027" exitCode=0
Jul 08 16:54:24.314872 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.314519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" event={"ID":"3cfc49f7-1e6d-4d50-b91d-d223d148ac28","Type":"ContainerDied","Data":"4352a3216b61fbf4c5ab108699b1401b684f8f1cca04275f3856eb7d02a25027"}
Jul 08 16:54:24.767018 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.766996    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:54:24.854124 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.854094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-proxy-tls\") pod \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") "
Jul 08 16:54:24.854301 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.854196    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-openshift-service-ca-bundle\") pod \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\" (UID: \"3cfc49f7-1e6d-4d50-b91d-d223d148ac28\") "
Jul 08 16:54:24.854548 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.854526    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3cfc49f7-1e6d-4d50-b91d-d223d148ac28" (UID: "3cfc49f7-1e6d-4d50-b91d-d223d148ac28"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:54:24.856217 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.856195    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3cfc49f7-1e6d-4d50-b91d-d223d148ac28" (UID: "3cfc49f7-1e6d-4d50-b91d-d223d148ac28"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:54:24.955003 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.954927    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:54:24.955003 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:24.954956    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3cfc49f7-1e6d-4d50-b91d-d223d148ac28-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:54:25.320421 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:25.320335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl" event={"ID":"3cfc49f7-1e6d-4d50-b91d-d223d148ac28","Type":"ContainerDied","Data":"e16fa96a7212b260e7330f187383854aca583e8252e342c87b129c57ef9674c4"}
Jul 08 16:54:25.320421 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:25.320372    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"
Jul 08 16:54:25.320421 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:25.320408    2562 scope.go:117] "RemoveContainer" containerID="4352a3216b61fbf4c5ab108699b1401b684f8f1cca04275f3856eb7d02a25027"
Jul 08 16:54:25.347211 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:25.347180    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"]
Jul 08 16:54:25.350564 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:25.350538    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8ceba-5774d86bd6-r5cbl"]
Jul 08 16:54:26.028250 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:26.028219    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" path="/var/lib/kubelet/pods/3cfc49f7-1e6d-4d50-b91d-d223d148ac28/volumes"
Jul 08 16:54:31.196855 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:31.196799    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 08 16:54:33.957311 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:33.957275    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"]
Jul 08 16:54:33.957695 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:33.957461    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" containerID="cri-o://ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae" gracePeriod=30
Jul 08 16:54:34.140157 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.140125    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"]
Jul 08 16:54:34.140386 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.140361    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" containerID="cri-o://b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc" gracePeriod=30
Jul 08 16:54:34.140518 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.140461    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kube-rbac-proxy" containerID="cri-o://4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465" gracePeriod=30
Jul 08 16:54:34.194705 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.194672    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"]
Jul 08 16:54:34.195065 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195048    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container"
Jul 08 16:54:34.195147 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195067    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container"
Jul 08 16:54:34.195147 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195080    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kube-rbac-proxy"
Jul 08 16:54:34.195147 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195087    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kube-rbac-proxy"
Jul 08 16:54:34.195147 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195096    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba"
Jul 08 16:54:34.195147 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195104    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba"
Jul 08 16:54:34.195388 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195203    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kube-rbac-proxy"
Jul 08 16:54:34.195388 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195217    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cfc49f7-1e6d-4d50-b91d-d223d148ac28" containerName="switch-graph-8ceba"
Jul 08 16:54:34.195388 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.195227    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5794e7e-1a94-4ccd-b00d-8d9adbea100a" containerName="kserve-container"
Jul 08 16:54:34.198372 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.198354    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.200494 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.200474    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3816e-predictor-serving-cert\""
Jul 08 16:54:34.200592 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.200578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3816e-kube-rbac-proxy-sar-config\""
Jul 08 16:54:34.211963 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.211907    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"]
Jul 08 16:54:34.342012 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.341980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57033467-18f8-48b4-b9b3-28a072b9b17c-proxy-tls\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.342012 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.342014    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpj7v\" (UniqueName: \"kubernetes.io/projected/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-api-access-vpj7v\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.342226 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.342151    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.358245 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.358215    2562 generic.go:356] "Generic (PLEG): container finished" podID="75e58397-959f-43bb-9709-4f1b09463274" containerID="4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465" exitCode=2
Jul 08 16:54:34.358373 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.358274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" event={"ID":"75e58397-959f-43bb-9709-4f1b09463274","Type":"ContainerDied","Data":"4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465"}
Jul 08 16:54:34.443548 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.443523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.443711 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.443583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57033467-18f8-48b4-b9b3-28a072b9b17c-proxy-tls\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.443711 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.443608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj7v\" (UniqueName: \"kubernetes.io/projected/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-api-access-vpj7v\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.444293 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.444267    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.446060 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.446040    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57033467-18f8-48b4-b9b3-28a072b9b17c-proxy-tls\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.452437 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.452418    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpj7v\" (UniqueName: \"kubernetes.io/projected/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-api-access-vpj7v\") pod \"success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") " pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.509550 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.509488    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:34.657224 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:34.657188    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"]
Jul 08 16:54:34.659799 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:54:34.659771    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57033467_18f8_48b4_b9b3_28a072b9b17c.slice/crio-d720026e43db8dcb00b45654660282eb99d6212320513c44cd3435e627f832ef WatchSource:0}: Error finding container d720026e43db8dcb00b45654660282eb99d6212320513c44cd3435e627f832ef: Status 404 returned error can't find the container with id d720026e43db8dcb00b45654660282eb99d6212320513c44cd3435e627f832ef
Jul 08 16:54:35.366571 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.366532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" event={"ID":"57033467-18f8-48b4-b9b3-28a072b9b17c","Type":"ContainerStarted","Data":"2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1"}
Jul 08 16:54:35.367098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.366609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" event={"ID":"57033467-18f8-48b4-b9b3-28a072b9b17c","Type":"ContainerStarted","Data":"e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094"}
Jul 08 16:54:35.367098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.366650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" event={"ID":"57033467-18f8-48b4-b9b3-28a072b9b17c","Type":"ContainerStarted","Data":"d720026e43db8dcb00b45654660282eb99d6212320513c44cd3435e627f832ef"}
Jul 08 16:54:35.367098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.366701    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:35.367098 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.366732    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:35.368077 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.368051    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 08 16:54:35.385091 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:35.385048    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podStartSLOduration=1.3850288069999999 podStartE2EDuration="1.385028807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:54:35.38332608 +0000 UTC m=+1295.945303474" watchObservedRunningTime="2026-07-08 16:54:35.385028807 +0000 UTC m=+1295.947006184"
Jul 08 16:54:36.371150 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:36.371104    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 08 16:54:37.038681 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.038633    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused"
Jul 08 16:54:37.044009 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.043974    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 08 16:54:37.291127 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.291045    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:37.825550 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.825525    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:54:37.972251 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.972156    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-642nz\" (UniqueName: \"kubernetes.io/projected/75e58397-959f-43bb-9709-4f1b09463274-kube-api-access-642nz\") pod \"75e58397-959f-43bb-9709-4f1b09463274\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") "
Jul 08 16:54:37.972251 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.972207    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/75e58397-959f-43bb-9709-4f1b09463274-kube-rbac-proxy-sar-config\") pod \"75e58397-959f-43bb-9709-4f1b09463274\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") "
Jul 08 16:54:37.972251 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.972239    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/75e58397-959f-43bb-9709-4f1b09463274-proxy-tls\") pod \"75e58397-959f-43bb-9709-4f1b09463274\" (UID: \"75e58397-959f-43bb-9709-4f1b09463274\") "
Jul 08 16:54:37.972594 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.972565    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e58397-959f-43bb-9709-4f1b09463274-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "75e58397-959f-43bb-9709-4f1b09463274" (UID: "75e58397-959f-43bb-9709-4f1b09463274"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:54:37.974763 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.974734    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e58397-959f-43bb-9709-4f1b09463274-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "75e58397-959f-43bb-9709-4f1b09463274" (UID: "75e58397-959f-43bb-9709-4f1b09463274"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:54:37.974763 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:37.974750    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e58397-959f-43bb-9709-4f1b09463274-kube-api-access-642nz" (OuterVolumeSpecName: "kube-api-access-642nz") pod "75e58397-959f-43bb-9709-4f1b09463274" (UID: "75e58397-959f-43bb-9709-4f1b09463274"). InnerVolumeSpecName "kube-api-access-642nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:54:38.073004 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.072968    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-642nz\" (UniqueName: \"kubernetes.io/projected/75e58397-959f-43bb-9709-4f1b09463274-kube-api-access-642nz\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:54:38.073004 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.073006    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/75e58397-959f-43bb-9709-4f1b09463274-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:54:38.073188 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.073023    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/75e58397-959f-43bb-9709-4f1b09463274-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:54:38.387314 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.387271    2562 generic.go:356] "Generic (PLEG): container finished" podID="75e58397-959f-43bb-9709-4f1b09463274" containerID="b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc" exitCode=0
Jul 08 16:54:38.387497 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.387408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" event={"ID":"75e58397-959f-43bb-9709-4f1b09463274","Type":"ContainerDied","Data":"b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc"}
Jul 08 16:54:38.387572 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.387501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw" event={"ID":"75e58397-959f-43bb-9709-4f1b09463274","Type":"ContainerDied","Data":"4cd1889a1e1eefe6b98ae4709509cea7161ad84e7b2a868dfbe901acac0144ff"}
Jul 08 16:54:38.387572 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.387543    2562 scope.go:117] "RemoveContainer" containerID="4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465"
Jul 08 16:54:38.387757 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.387735    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"
Jul 08 16:54:38.403252 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.402895    2562 scope.go:117] "RemoveContainer" containerID="b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc"
Jul 08 16:54:38.409639 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.409618    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"]
Jul 08 16:54:38.411440 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.411421    2562 scope.go:117] "RemoveContainer" containerID="4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465"
Jul 08 16:54:38.411715 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:54:38.411695    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465\": container with ID starting with 4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465 not found: ID does not exist" containerID="4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465"
Jul 08 16:54:38.411807 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.411730    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465"} err="failed to get container status \"4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465\": rpc error: code = NotFound desc = could not find container \"4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465\": container with ID starting with 4e35e225385ac4d1f971c6e016927b92da958529a59a9c3b96653d988aa0c465 not found: ID does not exist"
Jul 08 16:54:38.411807 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.411757    2562 scope.go:117] "RemoveContainer" containerID="b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc"
Jul 08 16:54:38.412049 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:54:38.412032    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc\": container with ID starting with b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc not found: ID does not exist" containerID="b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc"
Jul 08 16:54:38.412103 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.412057    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc"} err="failed to get container status \"b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc\": rpc error: code = NotFound desc = could not find container \"b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc\": container with ID starting with b73fcca51694b0e80b226e8cc18438b1f4e4f8fa47fabaee74715ccc488b43dc not found: ID does not exist"
Jul 08 16:54:38.416698 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:38.416677    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fabf2-predictor-7998766446-tv7kw"]
Jul 08 16:54:40.028908 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:40.028875    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e58397-959f-43bb-9709-4f1b09463274" path="/var/lib/kubelet/pods/75e58397-959f-43bb-9709-4f1b09463274/volumes"
Jul 08 16:54:41.196789 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:41.196758    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:54:41.376241 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:41.376210    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:54:41.376736 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:41.376706    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 08 16:54:42.290477 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:42.290437    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:47.291043 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:47.291000    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:47.291442 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:47.291108    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:54:51.377129 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:51.377084    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 08 16:54:52.291118 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:52.291076    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:54:54.379411 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.379371    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"]
Jul 08 16:54:54.379988 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.379880    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container"
Jul 08 16:54:54.379988 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.379903    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container"
Jul 08 16:54:54.379988 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.379918    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kube-rbac-proxy"
Jul 08 16:54:54.379988 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.379927    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kube-rbac-proxy"
Jul 08 16:54:54.380195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.380039    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kserve-container"
Jul 08 16:54:54.380195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.380056    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e58397-959f-43bb-9709-4f1b09463274" containerName="kube-rbac-proxy"
Jul 08 16:54:54.383053 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.383029    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.385652 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.385627    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-8befa-serving-cert\""
Jul 08 16:54:54.385786 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.385627    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-8befa-kube-rbac-proxy-sar-config\""
Jul 08 16:54:54.392782 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.392753    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"]
Jul 08 16:54:54.507300 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.507260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50981f56-610f-4f03-848f-5a37da7b8ad7-openshift-service-ca-bundle\") pod \"ensemble-graph-8befa-c7fbffddd-k7n7q\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") " pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.507300 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.507308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50981f56-610f-4f03-848f-5a37da7b8ad7-proxy-tls\") pod \"ensemble-graph-8befa-c7fbffddd-k7n7q\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") " pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.608769 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.608729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50981f56-610f-4f03-848f-5a37da7b8ad7-openshift-service-ca-bundle\") pod \"ensemble-graph-8befa-c7fbffddd-k7n7q\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") " pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.608769 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.608779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50981f56-610f-4f03-848f-5a37da7b8ad7-proxy-tls\") pod \"ensemble-graph-8befa-c7fbffddd-k7n7q\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") " pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.609462 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.609434    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50981f56-610f-4f03-848f-5a37da7b8ad7-openshift-service-ca-bundle\") pod \"ensemble-graph-8befa-c7fbffddd-k7n7q\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") " pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.611466 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.611444    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50981f56-610f-4f03-848f-5a37da7b8ad7-proxy-tls\") pod \"ensemble-graph-8befa-c7fbffddd-k7n7q\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") " pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.694343 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.694254    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:54.839078 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:54.839034    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"]
Jul 08 16:54:54.842613 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:54:54.842578    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50981f56_610f_4f03_848f_5a37da7b8ad7.slice/crio-dbac2d4eb6fdb5f5f32d12f3f01af22fcbd181f0d4bc4b0d7ed26ec9b6e5a0f5 WatchSource:0}: Error finding container dbac2d4eb6fdb5f5f32d12f3f01af22fcbd181f0d4bc4b0d7ed26ec9b6e5a0f5: Status 404 returned error can't find the container with id dbac2d4eb6fdb5f5f32d12f3f01af22fcbd181f0d4bc4b0d7ed26ec9b6e5a0f5
Jul 08 16:54:55.464727 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:55.464693    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" event={"ID":"50981f56-610f-4f03-848f-5a37da7b8ad7","Type":"ContainerStarted","Data":"e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e"}
Jul 08 16:54:55.465233 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:55.464756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" event={"ID":"50981f56-610f-4f03-848f-5a37da7b8ad7","Type":"ContainerStarted","Data":"dbac2d4eb6fdb5f5f32d12f3f01af22fcbd181f0d4bc4b0d7ed26ec9b6e5a0f5"}
Jul 08 16:54:55.465233 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:55.464797    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:54:55.482284 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:55.482199    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podStartSLOduration=1.482180141 podStartE2EDuration="1.482180141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:54:55.48121703 +0000 UTC m=+1316.043194415" watchObservedRunningTime="2026-07-08 16:54:55.482180141 +0000 UTC m=+1316.044157525"
Jul 08 16:54:57.290688 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:54:57.290638    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:01.377331 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:01.377293    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 08 16:55:01.477006 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:01.476969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:55:02.291538 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:02.291495    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:04.100034 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.100010    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:55:04.192973 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.192941    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a14cd999-092f-472b-97c7-563be16c2261-openshift-service-ca-bundle\") pod \"a14cd999-092f-472b-97c7-563be16c2261\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") "
Jul 08 16:55:04.192973 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.192977    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls\") pod \"a14cd999-092f-472b-97c7-563be16c2261\" (UID: \"a14cd999-092f-472b-97c7-563be16c2261\") "
Jul 08 16:55:04.193308 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.193283    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a14cd999-092f-472b-97c7-563be16c2261-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "a14cd999-092f-472b-97c7-563be16c2261" (UID: "a14cd999-092f-472b-97c7-563be16c2261"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:55:04.195100 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.195078    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a14cd999-092f-472b-97c7-563be16c2261" (UID: "a14cd999-092f-472b-97c7-563be16c2261"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:55:04.294059 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.293983    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a14cd999-092f-472b-97c7-563be16c2261-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:04.294059 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.294009    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a14cd999-092f-472b-97c7-563be16c2261-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:04.463232 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.463201    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"]
Jul 08 16:55:04.463449 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.463404    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" containerID="cri-o://e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e" gracePeriod=30
Jul 08 16:55:04.506759 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.506727    2562 generic.go:356] "Generic (PLEG): container finished" podID="a14cd999-092f-472b-97c7-563be16c2261" containerID="ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae" exitCode=0
Jul 08 16:55:04.506961 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.506777    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"
Jul 08 16:55:04.506961 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.506812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" event={"ID":"a14cd999-092f-472b-97c7-563be16c2261","Type":"ContainerDied","Data":"ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae"}
Jul 08 16:55:04.506961 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.506882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7" event={"ID":"a14cd999-092f-472b-97c7-563be16c2261","Type":"ContainerDied","Data":"56d1376826ccb28abe3faf2bd1ae2f54b53fe5567451b74130831f4ea9fc0e23"}
Jul 08 16:55:04.506961 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.506906    2562 scope.go:117] "RemoveContainer" containerID="ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae"
Jul 08 16:55:04.516207 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.516190    2562 scope.go:117] "RemoveContainer" containerID="ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae"
Jul 08 16:55:04.516477 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:55:04.516439    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae\": container with ID starting with ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae not found: ID does not exist" containerID="ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae"
Jul 08 16:55:04.516529 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.516485    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae"} err="failed to get container status \"ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae\": rpc error: code = NotFound desc = could not find container \"ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae\": container with ID starting with ceedbb2062e9c6d4258376c754ce0333ffd3617e5a511f21413863ee3e1c1bae not found: ID does not exist"
Jul 08 16:55:04.533248 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.533225    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"]
Jul 08 16:55:04.536548 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.536526    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fabf2-5f6647c4bf-kcmn7"]
Jul 08 16:55:04.602253 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.602217    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"]
Jul 08 16:55:04.602480 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.602452    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container" containerID="cri-o://4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef" gracePeriod=30
Jul 08 16:55:04.602549 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.602476    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kube-rbac-proxy" containerID="cri-o://053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e" gracePeriod=30
Jul 08 16:55:04.672502 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.672467    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"]
Jul 08 16:55:04.672795 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.672783    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2"
Jul 08 16:55:04.672861 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.672797    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2"
Jul 08 16:55:04.672901 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.672864    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a14cd999-092f-472b-97c7-563be16c2261" containerName="sequence-graph-fabf2"
Jul 08 16:55:04.677057 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.677038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.679837 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.679803    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-1f879-predictor-serving-cert\""
Jul 08 16:55:04.679924 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.679880    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-1f879-kube-rbac-proxy-sar-config\""
Jul 08 16:55:04.686647 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.686627    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"]
Jul 08 16:55:04.696830 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.696798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9t8j\" (UniqueName: \"kubernetes.io/projected/786486ea-9806-41d8-a8ca-8835eb3db625-kube-api-access-p9t8j\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.696929 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.696869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/786486ea-9806-41d8-a8ca-8835eb3db625-proxy-tls\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.696929 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.696924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/786486ea-9806-41d8-a8ca-8835eb3db625-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.797444 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.797415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9t8j\" (UniqueName: \"kubernetes.io/projected/786486ea-9806-41d8-a8ca-8835eb3db625-kube-api-access-p9t8j\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.797594 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.797454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/786486ea-9806-41d8-a8ca-8835eb3db625-proxy-tls\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.797663 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.797643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/786486ea-9806-41d8-a8ca-8835eb3db625-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.798338 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.798309    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/786486ea-9806-41d8-a8ca-8835eb3db625-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.799970 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.799950    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/786486ea-9806-41d8-a8ca-8835eb3db625-proxy-tls\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.811549 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.811524    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9t8j\" (UniqueName: \"kubernetes.io/projected/786486ea-9806-41d8-a8ca-8835eb3db625-kube-api-access-p9t8j\") pod \"success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") " pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:04.987929 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:04.987849    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:05.121733 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.121707    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"]
Jul 08 16:55:05.124410 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:55:05.124376    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786486ea_9806_41d8_a8ca_8835eb3db625.slice/crio-84c0ecc807d9d8231bea249fb0b43191d218cb09d4d39db2a35009546b0fa563 WatchSource:0}: Error finding container 84c0ecc807d9d8231bea249fb0b43191d218cb09d4d39db2a35009546b0fa563: Status 404 returned error can't find the container with id 84c0ecc807d9d8231bea249fb0b43191d218cb09d4d39db2a35009546b0fa563
Jul 08 16:55:05.513195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.513101    2562 generic.go:356] "Generic (PLEG): container finished" podID="c0601e45-bb2c-453e-9922-303007fe8fba" containerID="053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e" exitCode=2
Jul 08 16:55:05.513195 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.513179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" event={"ID":"c0601e45-bb2c-453e-9922-303007fe8fba","Type":"ContainerDied","Data":"053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e"}
Jul 08 16:55:05.515597 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.515576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" event={"ID":"786486ea-9806-41d8-a8ca-8835eb3db625","Type":"ContainerStarted","Data":"35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b"}
Jul 08 16:55:05.515696 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.515621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" event={"ID":"786486ea-9806-41d8-a8ca-8835eb3db625","Type":"ContainerStarted","Data":"075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7"}
Jul 08 16:55:05.515696 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.515644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" event={"ID":"786486ea-9806-41d8-a8ca-8835eb3db625","Type":"ContainerStarted","Data":"84c0ecc807d9d8231bea249fb0b43191d218cb09d4d39db2a35009546b0fa563"}
Jul 08 16:55:05.515849 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.515813    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:05.515954 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.515874    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:05.516968 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.516943    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 16:55:05.536524 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:05.536469    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podStartSLOduration=1.536451327 podStartE2EDuration="1.536451327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:55:05.534410281 +0000 UTC m=+1326.096387673" watchObservedRunningTime="2026-07-08 16:55:05.536451327 +0000 UTC m=+1326.098428712"
Jul 08 16:55:06.029352 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:06.029308    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a14cd999-092f-472b-97c7-563be16c2261" path="/var/lib/kubelet/pods/a14cd999-092f-472b-97c7-563be16c2261/volumes"
Jul 08 16:55:06.191420 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:06.191377    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Jul 08 16:55:06.473312 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:06.473266    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:06.520273 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:06.520233    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 16:55:07.956566 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:07.956541    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:55:08.021950 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.021877    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7rb6\" (UniqueName: \"kubernetes.io/projected/c0601e45-bb2c-453e-9922-303007fe8fba-kube-api-access-z7rb6\") pod \"c0601e45-bb2c-453e-9922-303007fe8fba\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") "
Jul 08 16:55:08.021950 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.021915    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0601e45-bb2c-453e-9922-303007fe8fba-kube-rbac-proxy-sar-config\") pod \"c0601e45-bb2c-453e-9922-303007fe8fba\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") "
Jul 08 16:55:08.022145 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.021960    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0601e45-bb2c-453e-9922-303007fe8fba-proxy-tls\") pod \"c0601e45-bb2c-453e-9922-303007fe8fba\" (UID: \"c0601e45-bb2c-453e-9922-303007fe8fba\") "
Jul 08 16:55:08.022318 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.022294    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0601e45-bb2c-453e-9922-303007fe8fba-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c0601e45-bb2c-453e-9922-303007fe8fba" (UID: "c0601e45-bb2c-453e-9922-303007fe8fba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:55:08.024055 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.024032    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0601e45-bb2c-453e-9922-303007fe8fba-kube-api-access-z7rb6" (OuterVolumeSpecName: "kube-api-access-z7rb6") pod "c0601e45-bb2c-453e-9922-303007fe8fba" (UID: "c0601e45-bb2c-453e-9922-303007fe8fba"). InnerVolumeSpecName "kube-api-access-z7rb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:55:08.024158 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.024079    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0601e45-bb2c-453e-9922-303007fe8fba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c0601e45-bb2c-453e-9922-303007fe8fba" (UID: "c0601e45-bb2c-453e-9922-303007fe8fba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:55:08.122683 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.122656    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7rb6\" (UniqueName: \"kubernetes.io/projected/c0601e45-bb2c-453e-9922-303007fe8fba-kube-api-access-z7rb6\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:08.122683 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.122682    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0601e45-bb2c-453e-9922-303007fe8fba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:08.122879 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.122692    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0601e45-bb2c-453e-9922-303007fe8fba-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:08.529748 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.529709    2562 generic.go:356] "Generic (PLEG): container finished" podID="c0601e45-bb2c-453e-9922-303007fe8fba" containerID="4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef" exitCode=0
Jul 08 16:55:08.529936 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.529775    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"
Jul 08 16:55:08.529936 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.529786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" event={"ID":"c0601e45-bb2c-453e-9922-303007fe8fba","Type":"ContainerDied","Data":"4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef"}
Jul 08 16:55:08.529936 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.529869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s" event={"ID":"c0601e45-bb2c-453e-9922-303007fe8fba","Type":"ContainerDied","Data":"21f5800423e166211f83aeb0b9d115a7f2b3637eb1064d5c3c24db9f431d4afd"}
Jul 08 16:55:08.529936 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.529896    2562 scope.go:117] "RemoveContainer" containerID="053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e"
Jul 08 16:55:08.538964 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.538900    2562 scope.go:117] "RemoveContainer" containerID="4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef"
Jul 08 16:55:08.546013 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.545994    2562 scope.go:117] "RemoveContainer" containerID="053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e"
Jul 08 16:55:08.546249 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:55:08.546232    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e\": container with ID starting with 053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e not found: ID does not exist" containerID="053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e"
Jul 08 16:55:08.546294 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.546257    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e"} err="failed to get container status \"053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e\": rpc error: code = NotFound desc = could not find container \"053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e\": container with ID starting with 053e01acd5175fd22af7e579bb643abcac510eaa66dfdb7b1521a64859e66a2e not found: ID does not exist"
Jul 08 16:55:08.546294 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.546273    2562 scope.go:117] "RemoveContainer" containerID="4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef"
Jul 08 16:55:08.546476 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:55:08.546462    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef\": container with ID starting with 4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef not found: ID does not exist" containerID="4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef"
Jul 08 16:55:08.546525 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.546478    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef"} err="failed to get container status \"4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef\": rpc error: code = NotFound desc = could not find container \"4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef\": container with ID starting with 4a22aa11864fae52e233eb58508051f80b6118b16bbed93531941c872406a3ef not found: ID does not exist"
Jul 08 16:55:08.549184 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.549163    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"]
Jul 08 16:55:08.554996 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:08.554976    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8befa-predictor-85866c45b8-tgk8s"]
Jul 08 16:55:10.029290 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:10.029255    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" path="/var/lib/kubelet/pods/c0601e45-bb2c-453e-9922-303007fe8fba/volumes"
Jul 08 16:55:11.377379 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:11.377345    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 08 16:55:11.473985 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:11.473952    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:11.525116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:11.525088    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:11.525552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:11.525529    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 16:55:16.474149 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:16.474105    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:16.474551 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:16.474219    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:55:21.377515 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:21.377479    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:55:21.473327 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:21.473291    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:21.525605 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:21.525564    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 16:55:26.473883 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:26.473783    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:31.474171 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:31.474130    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:31.525956 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:31.525917    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 16:55:34.211585 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.211549    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"]
Jul 08 16:55:34.212116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.212002    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container"
Jul 08 16:55:34.212116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.212018    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container"
Jul 08 16:55:34.212116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.212037    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kube-rbac-proxy"
Jul 08 16:55:34.212116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.212043    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kube-rbac-proxy"
Jul 08 16:55:34.212116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.212101    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kube-rbac-proxy"
Jul 08 16:55:34.212116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.212113    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0601e45-bb2c-453e-9922-303007fe8fba" containerName="kserve-container"
Jul 08 16:55:34.214997 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.214978    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.217527 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.217504    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-3816e-kube-rbac-proxy-sar-config\""
Jul 08 16:55:34.217640 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.217503    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-3816e-serving-cert\""
Jul 08 16:55:34.225937 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.225908    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"]
Jul 08 16:55:34.335969 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.335925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-proxy-tls\") pod \"sequence-graph-3816e-6d744db655-797vb\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") " pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.336149 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.335991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-openshift-service-ca-bundle\") pod \"sequence-graph-3816e-6d744db655-797vb\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") " pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.436930 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.436891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-openshift-service-ca-bundle\") pod \"sequence-graph-3816e-6d744db655-797vb\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") " pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.437108 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.436968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-proxy-tls\") pod \"sequence-graph-3816e-6d744db655-797vb\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") " pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.437778 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.437750    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-openshift-service-ca-bundle\") pod \"sequence-graph-3816e-6d744db655-797vb\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") " pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.439373 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.439354    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-proxy-tls\") pod \"sequence-graph-3816e-6d744db655-797vb\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") " pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.528283 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.528249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:34.616656 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.616632    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:55:34.644785 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.644745    2562 generic.go:356] "Generic (PLEG): container finished" podID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerID="e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e" exitCode=0
Jul 08 16:55:34.644969 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.644790    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"
Jul 08 16:55:34.644969 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.644844    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" event={"ID":"50981f56-610f-4f03-848f-5a37da7b8ad7","Type":"ContainerDied","Data":"e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e"}
Jul 08 16:55:34.644969 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.644910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q" event={"ID":"50981f56-610f-4f03-848f-5a37da7b8ad7","Type":"ContainerDied","Data":"dbac2d4eb6fdb5f5f32d12f3f01af22fcbd181f0d4bc4b0d7ed26ec9b6e5a0f5"}
Jul 08 16:55:34.644969 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.644941    2562 scope.go:117] "RemoveContainer" containerID="e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e"
Jul 08 16:55:34.654353 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.654333    2562 scope.go:117] "RemoveContainer" containerID="e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e"
Jul 08 16:55:34.654631 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:55:34.654611    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e\": container with ID starting with e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e not found: ID does not exist" containerID="e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e"
Jul 08 16:55:34.654735 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.654642    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e"} err="failed to get container status \"e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e\": rpc error: code = NotFound desc = could not find container \"e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e\": container with ID starting with e223ad8b2abbe2f68384f4b9da6d005ce83aad7874c3d44a8ffbe6012b9eec5e not found: ID does not exist"
Jul 08 16:55:34.679242 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.679220    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"]
Jul 08 16:55:34.682765 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:55:34.682740    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c00ae3b_6cda_471d_9fcf_c8245a7ae654.slice/crio-e59b4560f5691cd0f15e7d5c89064baaf12cd107da7f06f0007c9fa9724ffcf3 WatchSource:0}: Error finding container e59b4560f5691cd0f15e7d5c89064baaf12cd107da7f06f0007c9fa9724ffcf3: Status 404 returned error can't find the container with id e59b4560f5691cd0f15e7d5c89064baaf12cd107da7f06f0007c9fa9724ffcf3
Jul 08 16:55:34.740272 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.740251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50981f56-610f-4f03-848f-5a37da7b8ad7-proxy-tls\") pod \"50981f56-610f-4f03-848f-5a37da7b8ad7\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") "
Jul 08 16:55:34.740387 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.740286    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50981f56-610f-4f03-848f-5a37da7b8ad7-openshift-service-ca-bundle\") pod \"50981f56-610f-4f03-848f-5a37da7b8ad7\" (UID: \"50981f56-610f-4f03-848f-5a37da7b8ad7\") "
Jul 08 16:55:34.740631 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.740611    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50981f56-610f-4f03-848f-5a37da7b8ad7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "50981f56-610f-4f03-848f-5a37da7b8ad7" (UID: "50981f56-610f-4f03-848f-5a37da7b8ad7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:55:34.742151 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.742131    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50981f56-610f-4f03-848f-5a37da7b8ad7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50981f56-610f-4f03-848f-5a37da7b8ad7" (UID: "50981f56-610f-4f03-848f-5a37da7b8ad7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:55:34.841662 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.841624    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50981f56-610f-4f03-848f-5a37da7b8ad7-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:34.841662 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.841658    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50981f56-610f-4f03-848f-5a37da7b8ad7-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:34.968048 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.967982    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"]
Jul 08 16:55:34.972061 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:34.972036    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8befa-c7fbffddd-k7n7q"]
Jul 08 16:55:35.650983 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:35.650947    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" event={"ID":"8c00ae3b-6cda-471d-9fcf-c8245a7ae654","Type":"ContainerStarted","Data":"4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214"}
Jul 08 16:55:35.651353 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:35.651001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" event={"ID":"8c00ae3b-6cda-471d-9fcf-c8245a7ae654","Type":"ContainerStarted","Data":"e59b4560f5691cd0f15e7d5c89064baaf12cd107da7f06f0007c9fa9724ffcf3"}
Jul 08 16:55:35.651353 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:35.651033    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:35.668160 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:35.668118    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podStartSLOduration=1.668105507 podStartE2EDuration="1.668105507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:55:35.667173112 +0000 UTC m=+1356.229150495" watchObservedRunningTime="2026-07-08 16:55:35.668105507 +0000 UTC m=+1356.230082890"
Jul 08 16:55:36.029529 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:36.029499    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" path="/var/lib/kubelet/pods/50981f56-610f-4f03-848f-5a37da7b8ad7/volumes"
Jul 08 16:55:41.525632 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:41.525594    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 16:55:41.666977 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:41.666949    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:55:44.286765 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.286729    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"]
Jul 08 16:55:44.287181 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.286944    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" containerID="cri-o://4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214" gracePeriod=30
Jul 08 16:55:44.399175 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.399145    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"]
Jul 08 16:55:44.399469 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.399420    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container" containerID="cri-o://e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094" gracePeriod=30
Jul 08 16:55:44.399528 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.399468    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kube-rbac-proxy" containerID="cri-o://2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1" gracePeriod=30
Jul 08 16:55:44.488246 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.488214    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"]
Jul 08 16:55:44.488535 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.488523    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa"
Jul 08 16:55:44.488590 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.488536    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa"
Jul 08 16:55:44.488641 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.488599    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="50981f56-610f-4f03-848f-5a37da7b8ad7" containerName="ensemble-graph-8befa"
Jul 08 16:55:44.492871 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.492855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.495757 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.495740    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f6dd6-predictor-serving-cert\""
Jul 08 16:55:44.495859 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.495741    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f6dd6-kube-rbac-proxy-sar-config\""
Jul 08 16:55:44.506848 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.506801    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"]
Jul 08 16:55:44.621893 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.621859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee220760-6390-446a-b641-f2bfce2f1c52-proxy-tls\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.622075 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.621935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee220760-6390-446a-b641-f2bfce2f1c52-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.622075 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.621972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-626dp\" (UniqueName: \"kubernetes.io/projected/ee220760-6390-446a-b641-f2bfce2f1c52-kube-api-access-626dp\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.696167 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.696134    2562 generic.go:356] "Generic (PLEG): container finished" podID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerID="2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1" exitCode=2
Jul 08 16:55:44.696341 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.696207    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" event={"ID":"57033467-18f8-48b4-b9b3-28a072b9b17c","Type":"ContainerDied","Data":"2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1"}
Jul 08 16:55:44.722670 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.722637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-626dp\" (UniqueName: \"kubernetes.io/projected/ee220760-6390-446a-b641-f2bfce2f1c52-kube-api-access-626dp\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.722806 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.722709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee220760-6390-446a-b641-f2bfce2f1c52-proxy-tls\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.722806 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.722749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee220760-6390-446a-b641-f2bfce2f1c52-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.723406 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.723382    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee220760-6390-446a-b641-f2bfce2f1c52-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.725313 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.725290    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee220760-6390-446a-b641-f2bfce2f1c52-proxy-tls\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.731792 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.731766    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-626dp\" (UniqueName: \"kubernetes.io/projected/ee220760-6390-446a-b641-f2bfce2f1c52-kube-api-access-626dp\") pod \"success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") " pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.802885 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.802853    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:44.937343 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:44.937315    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"]
Jul 08 16:55:44.939534 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:55:44.939509    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee220760_6390_446a_b641_f2bfce2f1c52.slice/crio-db4dd33afd80185723bc36e0d1cf2d42d77dbae9bdd79851cc5611c553b1e047 WatchSource:0}: Error finding container db4dd33afd80185723bc36e0d1cf2d42d77dbae9bdd79851cc5611c553b1e047: Status 404 returned error can't find the container with id db4dd33afd80185723bc36e0d1cf2d42d77dbae9bdd79851cc5611c553b1e047
Jul 08 16:55:45.702794 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:45.702760    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" event={"ID":"ee220760-6390-446a-b641-f2bfce2f1c52","Type":"ContainerStarted","Data":"8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713"}
Jul 08 16:55:45.703218 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:45.702816    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" event={"ID":"ee220760-6390-446a-b641-f2bfce2f1c52","Type":"ContainerStarted","Data":"ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c"}
Jul 08 16:55:45.703218 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:45.702849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" event={"ID":"ee220760-6390-446a-b641-f2bfce2f1c52","Type":"ContainerStarted","Data":"db4dd33afd80185723bc36e0d1cf2d42d77dbae9bdd79851cc5611c553b1e047"}
Jul 08 16:55:45.703218 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:45.702890    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:45.721167 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:45.721121    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podStartSLOduration=1.721107016 podStartE2EDuration="1.721107016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:55:45.719760351 +0000 UTC m=+1366.281737735" watchObservedRunningTime="2026-07-08 16:55:45.721107016 +0000 UTC m=+1366.283084399"
Jul 08 16:55:46.371375 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:46.371336    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 08 16:55:46.666142 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:46.666059    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:46.707669 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:46.707641    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:46.709037 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:46.709009    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 08 16:55:47.661187 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.661157    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:55:47.713382 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.713302    2562 generic.go:356] "Generic (PLEG): container finished" podID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerID="e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094" exitCode=0
Jul 08 16:55:47.713382 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.713353    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"
Jul 08 16:55:47.713779 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.713391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" event={"ID":"57033467-18f8-48b4-b9b3-28a072b9b17c","Type":"ContainerDied","Data":"e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094"}
Jul 08 16:55:47.713779 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.713454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d" event={"ID":"57033467-18f8-48b4-b9b3-28a072b9b17c","Type":"ContainerDied","Data":"d720026e43db8dcb00b45654660282eb99d6212320513c44cd3435e627f832ef"}
Jul 08 16:55:47.713779 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.713484    2562 scope.go:117] "RemoveContainer" containerID="2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1"
Jul 08 16:55:47.713936 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.713904    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 08 16:55:47.721264 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.721246    2562 scope.go:117] "RemoveContainer" containerID="e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094"
Jul 08 16:55:47.728463 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.728447    2562 scope.go:117] "RemoveContainer" containerID="2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1"
Jul 08 16:55:47.728758 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:55:47.728740    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1\": container with ID starting with 2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1 not found: ID does not exist" containerID="2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1"
Jul 08 16:55:47.728806 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.728767    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1"} err="failed to get container status \"2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1\": rpc error: code = NotFound desc = could not find container \"2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1\": container with ID starting with 2caf16415aeeda36e0cbdbb3d35ca9cb549558cf42e3731a6fd7b0f6f5b61ed1 not found: ID does not exist"
Jul 08 16:55:47.728806 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.728785    2562 scope.go:117] "RemoveContainer" containerID="e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094"
Jul 08 16:55:47.729048 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:55:47.729026    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094\": container with ID starting with e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094 not found: ID does not exist" containerID="e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094"
Jul 08 16:55:47.729132 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.729058    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094"} err="failed to get container status \"e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094\": rpc error: code = NotFound desc = could not find container \"e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094\": container with ID starting with e325705f31863b5f8678ea7cc8ed7aec350b6745b9b0a3ab83a3e0519bee2094 not found: ID does not exist"
Jul 08 16:55:47.750340 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.750318    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-rbac-proxy-sar-config\") pod \"57033467-18f8-48b4-b9b3-28a072b9b17c\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") "
Jul 08 16:55:47.750429 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.750371    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpj7v\" (UniqueName: \"kubernetes.io/projected/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-api-access-vpj7v\") pod \"57033467-18f8-48b4-b9b3-28a072b9b17c\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") "
Jul 08 16:55:47.750465 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.750431    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57033467-18f8-48b4-b9b3-28a072b9b17c-proxy-tls\") pod \"57033467-18f8-48b4-b9b3-28a072b9b17c\" (UID: \"57033467-18f8-48b4-b9b3-28a072b9b17c\") "
Jul 08 16:55:47.750668 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.750644    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "57033467-18f8-48b4-b9b3-28a072b9b17c" (UID: "57033467-18f8-48b4-b9b3-28a072b9b17c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:55:47.752316 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.752287    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-api-access-vpj7v" (OuterVolumeSpecName: "kube-api-access-vpj7v") pod "57033467-18f8-48b4-b9b3-28a072b9b17c" (UID: "57033467-18f8-48b4-b9b3-28a072b9b17c"). InnerVolumeSpecName "kube-api-access-vpj7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:55:47.752404 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.752380    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57033467-18f8-48b4-b9b3-28a072b9b17c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "57033467-18f8-48b4-b9b3-28a072b9b17c" (UID: "57033467-18f8-48b4-b9b3-28a072b9b17c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:55:47.851451 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.851421    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57033467-18f8-48b4-b9b3-28a072b9b17c-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:47.851451 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.851453    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:47.851690 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:47.851467    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpj7v\" (UniqueName: \"kubernetes.io/projected/57033467-18f8-48b4-b9b3-28a072b9b17c-kube-api-access-vpj7v\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:55:48.036340 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:48.036313    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"]
Jul 08 16:55:48.040211 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:48.040185    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3816e-predictor-5ff79f6dcd-g5g6d"]
Jul 08 16:55:50.029320 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:50.029277    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" path="/var/lib/kubelet/pods/57033467-18f8-48b4-b9b3-28a072b9b17c/volumes"
Jul 08 16:55:51.526840 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:51.526796    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 16:55:51.666116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:51.666084    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:52.718115 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:52.718087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:55:52.718484 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:52.718445    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 08 16:55:56.666094 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:56.666053    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:55:56.666535 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:55:56.666174    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:56:01.665502 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:01.665461    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:56:02.718696 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:02.718655    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 08 16:56:04.724107 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724075    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"]
Jul 08 16:56:04.724468 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724378    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container"
Jul 08 16:56:04.724468 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724388    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container"
Jul 08 16:56:04.724468 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724406    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kube-rbac-proxy"
Jul 08 16:56:04.724468 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724413    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kube-rbac-proxy"
Jul 08 16:56:04.724468 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724467    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kserve-container"
Jul 08 16:56:04.724665 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.724475    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="57033467-18f8-48b4-b9b3-28a072b9b17c" containerName="kube-rbac-proxy"
Jul 08 16:56:04.728859 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.728840    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:04.731563 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.731540    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1f879-serving-cert\""
Jul 08 16:56:04.731702 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.731540    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1f879-kube-rbac-proxy-sar-config\""
Jul 08 16:56:04.736712 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.736692    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"]
Jul 08 16:56:04.784512 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.784490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be147abb-e4e1-4eef-9ed1-1e4162a088ca-openshift-service-ca-bundle\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:04.784628 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.784527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:04.885566 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.885531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be147abb-e4e1-4eef-9ed1-1e4162a088ca-openshift-service-ca-bundle\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:04.885737 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.885582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:04.885737 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:56:04.885711    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-1f879-serving-cert: secret "ensemble-graph-1f879-serving-cert" not found
Jul 08 16:56:04.885845 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:56:04.885768    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls podName:be147abb-e4e1-4eef-9ed1-1e4162a088ca nodeName:}" failed. No retries permitted until 2026-07-08 16:56:05.385751474 +0000 UTC m=+1385.947728852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls") pod "ensemble-graph-1f879-76cbbb6865-l8c8g" (UID: "be147abb-e4e1-4eef-9ed1-1e4162a088ca") : secret "ensemble-graph-1f879-serving-cert" not found
Jul 08 16:56:04.886156 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:04.886138    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be147abb-e4e1-4eef-9ed1-1e4162a088ca-openshift-service-ca-bundle\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:05.388340 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:05.388305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:05.390635 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:05.390615    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls\") pod \"ensemble-graph-1f879-76cbbb6865-l8c8g\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") " pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:05.641541 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:05.641460    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:05.766746 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:05.766713    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"]
Jul 08 16:56:05.769224 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:56:05.769199    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe147abb_e4e1_4eef_9ed1_1e4162a088ca.slice/crio-8e76e3a20e7a8f58f0a27186b4f576310978682e7971f5c8476eafe58679f8f3 WatchSource:0}: Error finding container 8e76e3a20e7a8f58f0a27186b4f576310978682e7971f5c8476eafe58679f8f3: Status 404 returned error can't find the container with id 8e76e3a20e7a8f58f0a27186b4f576310978682e7971f5c8476eafe58679f8f3
Jul 08 16:56:05.799767 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:05.799742    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" event={"ID":"be147abb-e4e1-4eef-9ed1-1e4162a088ca","Type":"ContainerStarted","Data":"8e76e3a20e7a8f58f0a27186b4f576310978682e7971f5c8476eafe58679f8f3"}
Jul 08 16:56:06.665615 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:06.665568    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:56:06.805689 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:06.805652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" event={"ID":"be147abb-e4e1-4eef-9ed1-1e4162a088ca","Type":"ContainerStarted","Data":"ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07"}
Jul 08 16:56:06.806078 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:06.805736    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:06.822417 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:06.822372    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podStartSLOduration=2.822358844 podStartE2EDuration="2.822358844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:06.822281751 +0000 UTC m=+1387.384259148" watchObservedRunningTime="2026-07-08 16:56:06.822358844 +0000 UTC m=+1387.384336210"
Jul 08 16:56:11.665772 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:11.665733    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:56:12.718563 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:12.718524    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 08 16:56:12.816264 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:12.816238    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 16:56:14.456116 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.456090    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:56:14.550136 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.550053    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-openshift-service-ca-bundle\") pod \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") "
Jul 08 16:56:14.550136 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.550100    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-proxy-tls\") pod \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\" (UID: \"8c00ae3b-6cda-471d-9fcf-c8245a7ae654\") "
Jul 08 16:56:14.550409 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.550385    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8c00ae3b-6cda-471d-9fcf-c8245a7ae654" (UID: "8c00ae3b-6cda-471d-9fcf-c8245a7ae654"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:56:14.552142 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.552116    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8c00ae3b-6cda-471d-9fcf-c8245a7ae654" (UID: "8c00ae3b-6cda-471d-9fcf-c8245a7ae654"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:56:14.650695 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.650655    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:56:14.650695 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.650690    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c00ae3b-6cda-471d-9fcf-c8245a7ae654-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 16:56:14.842474 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.842436    2562 generic.go:356] "Generic (PLEG): container finished" podID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerID="4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214" exitCode=0
Jul 08 16:56:14.842629 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.842483    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"
Jul 08 16:56:14.842629 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.842486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" event={"ID":"8c00ae3b-6cda-471d-9fcf-c8245a7ae654","Type":"ContainerDied","Data":"4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214"}
Jul 08 16:56:14.842629 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.842538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb" event={"ID":"8c00ae3b-6cda-471d-9fcf-c8245a7ae654","Type":"ContainerDied","Data":"e59b4560f5691cd0f15e7d5c89064baaf12cd107da7f06f0007c9fa9724ffcf3"}
Jul 08 16:56:14.842629 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.842567    2562 scope.go:117] "RemoveContainer" containerID="4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214"
Jul 08 16:56:14.851492 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.851475    2562 scope.go:117] "RemoveContainer" containerID="4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214"
Jul 08 16:56:14.851737 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:56:14.851721    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214\": container with ID starting with 4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214 not found: ID does not exist" containerID="4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214"
Jul 08 16:56:14.851807 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.851744    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214"} err="failed to get container status \"4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214\": rpc error: code = NotFound desc = could not find container \"4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214\": container with ID starting with 4595fe515e3afee3fad8b98b1d36d3d3a2d017ef008cc4ecb4cb69bc3a453214 not found: ID does not exist"
Jul 08 16:56:14.863783 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.863761    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"]
Jul 08 16:56:14.867552 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:14.867529    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3816e-6d744db655-797vb"]
Jul 08 16:56:16.028355 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:16.028324    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" path="/var/lib/kubelet/pods/8c00ae3b-6cda-471d-9fcf-c8245a7ae654/volumes"
Jul 08 16:56:22.718638 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:22.718598    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 08 16:56:32.719982 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:32.719953    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 16:56:44.548776 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.548745    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"]
Jul 08 16:56:44.549314 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.549124    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e"
Jul 08 16:56:44.549314 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.549136    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e"
Jul 08 16:56:44.549314 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.549198    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c00ae3b-6cda-471d-9fcf-c8245a7ae654" containerName="sequence-graph-3816e"
Jul 08 16:56:44.552136 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.552120    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:44.554459 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.554438    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-f6dd6-kube-rbac-proxy-sar-config\""
Jul 08 16:56:44.555376 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.555355    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-f6dd6-serving-cert\""
Jul 08 16:56:44.561718 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.561694    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"]
Jul 08 16:56:44.579651 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.579631    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c532c61d-3d07-47d9-bad2-19801079abab-openshift-service-ca-bundle\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:44.579739 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.579664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:44.681080 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.681048    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c532c61d-3d07-47d9-bad2-19801079abab-openshift-service-ca-bundle\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:44.681260 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.681087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:44.681260 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:56:44.681220    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-f6dd6-serving-cert: secret "sequence-graph-f6dd6-serving-cert" not found
Jul 08 16:56:44.681388 ip-10-0-132-182 kubenswrapper[2562]: E0708 16:56:44.681288    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls podName:c532c61d-3d07-47d9-bad2-19801079abab nodeName:}" failed. No retries permitted until 2026-07-08 16:56:45.181268199 +0000 UTC m=+1425.743245578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls") pod "sequence-graph-f6dd6-cbdd594c5-jgpqf" (UID: "c532c61d-3d07-47d9-bad2-19801079abab") : secret "sequence-graph-f6dd6-serving-cert" not found
Jul 08 16:56:44.681734 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:44.681709    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c532c61d-3d07-47d9-bad2-19801079abab-openshift-service-ca-bundle\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:45.184666 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.184616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:45.187062 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.187041    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls\") pod \"sequence-graph-f6dd6-cbdd594c5-jgpqf\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") " pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:45.463592 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.463494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:45.603899 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.603875    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"]
Jul 08 16:56:45.606076 ip-10-0-132-182 kubenswrapper[2562]: W0708 16:56:45.606039    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc532c61d_3d07_47d9_bad2_19801079abab.slice/crio-a5ff3755e56cd752c3c6e5a837a876a8d052aadbe5898c96ccbefd50da1549f0 WatchSource:0}: Error finding container a5ff3755e56cd752c3c6e5a837a876a8d052aadbe5898c96ccbefd50da1549f0: Status 404 returned error can't find the container with id a5ff3755e56cd752c3c6e5a837a876a8d052aadbe5898c96ccbefd50da1549f0
Jul 08 16:56:45.978869 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.978832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" event={"ID":"c532c61d-3d07-47d9-bad2-19801079abab","Type":"ContainerStarted","Data":"fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c"}
Jul 08 16:56:45.979084 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.978888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" event={"ID":"c532c61d-3d07-47d9-bad2-19801079abab","Type":"ContainerStarted","Data":"a5ff3755e56cd752c3c6e5a837a876a8d052aadbe5898c96ccbefd50da1549f0"}
Jul 08 16:56:45.979084 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.978951    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:56:45.997230 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:45.997162    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podStartSLOduration=1.99714722 podStartE2EDuration="1.99714722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:45.99626286 +0000 UTC m=+1426.558240248" watchObservedRunningTime="2026-07-08 16:56:45.99714722 +0000 UTC m=+1426.559124603"
Jul 08 16:56:51.987980 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:56:51.987946    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 16:58:03.349031 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:58:03.349000    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 16:58:03.351194 ip-10-0-132-182 kubenswrapper[2562]: I0708 16:58:03.351171    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:03:03.374133 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:03:03.374099    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:03:03.376791 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:03:03.376770    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:04:19.314847 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.314798    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"]
Jul 08 17:04:19.315719 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.315018    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" containerID="cri-o://ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07" gracePeriod=30
Jul 08 17:04:19.458077 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.458043    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"]
Jul 08 17:04:19.458388 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.458328    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" containerID="cri-o://075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7" gracePeriod=30
Jul 08 17:04:19.458388 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.458364    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kube-rbac-proxy" containerID="cri-o://35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b" gracePeriod=30
Jul 08 17:04:19.534911 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.534877    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"]
Jul 08 17:04:19.544262 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.544240    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.548425 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.548015    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-78c4a-predictor-serving-cert\""
Jul 08 17:04:19.548425 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.548282    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-78c4a-kube-rbac-proxy-sar-config\""
Jul 08 17:04:19.550138 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.550115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"]
Jul 08 17:04:19.603783 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.603755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.603950 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.603786    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gkvg\" (UniqueName: \"kubernetes.io/projected/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-api-access-2gkvg\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.603950 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.603812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.705037 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.704997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.705037 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.705042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gkvg\" (UniqueName: \"kubernetes.io/projected/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-api-access-2gkvg\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.705304 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.705079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.705304 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:19.705247    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-serving-cert: secret "success-200-isvc-78c4a-predictor-serving-cert" not found
Jul 08 17:04:19.705422 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:19.705311    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls podName:1389556b-cd1b-4312-8ed0-798eb53093b4 nodeName:}" failed. No retries permitted until 2026-07-08 17:04:20.205291936 +0000 UTC m=+1880.767269301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls") pod "success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" (UID: "1389556b-cd1b-4312-8ed0-798eb53093b4") : secret "success-200-isvc-78c4a-predictor-serving-cert" not found
Jul 08 17:04:19.705677 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.705657    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.714670 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.714640    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gkvg\" (UniqueName: \"kubernetes.io/projected/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-api-access-2gkvg\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:19.896623 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.896535    2562 generic.go:356] "Generic (PLEG): container finished" podID="786486ea-9806-41d8-a8ca-8835eb3db625" containerID="35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b" exitCode=2
Jul 08 17:04:19.896623 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:19.896581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" event={"ID":"786486ea-9806-41d8-a8ca-8835eb3db625","Type":"ContainerDied","Data":"35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b"}
Jul 08 17:04:20.209144 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:20.209056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:20.209298 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:20.209228    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-serving-cert: secret "success-200-isvc-78c4a-predictor-serving-cert" not found
Jul 08 17:04:20.209337 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:20.209325    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls podName:1389556b-cd1b-4312-8ed0-798eb53093b4 nodeName:}" failed. No retries permitted until 2026-07-08 17:04:21.209295784 +0000 UTC m=+1881.771273152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls") pod "success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" (UID: "1389556b-cd1b-4312-8ed0-798eb53093b4") : secret "success-200-isvc-78c4a-predictor-serving-cert" not found
Jul 08 17:04:21.217633 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.217589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:21.220069 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.220046    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") pod \"success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") " pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:21.362866 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.362810    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:21.500695 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.500670    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"]
Jul 08 17:04:21.503234 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:04:21.503200    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1389556b_cd1b_4312_8ed0_798eb53093b4.slice/crio-2f63f12404083f1a4b96a9c942af1ccfa015ae8f1d6df9bdf6f7b33520903848 WatchSource:0}: Error finding container 2f63f12404083f1a4b96a9c942af1ccfa015ae8f1d6df9bdf6f7b33520903848: Status 404 returned error can't find the container with id 2f63f12404083f1a4b96a9c942af1ccfa015ae8f1d6df9bdf6f7b33520903848
Jul 08 17:04:21.521526 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.521481    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Jul 08 17:04:21.525990 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.525964    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 08 17:04:21.908313 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.908272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" event={"ID":"1389556b-cd1b-4312-8ed0-798eb53093b4","Type":"ContainerStarted","Data":"24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a"}
Jul 08 17:04:21.908527 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.908339    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:21.908527 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.908360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" event={"ID":"1389556b-cd1b-4312-8ed0-798eb53093b4","Type":"ContainerStarted","Data":"92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161"}
Jul 08 17:04:21.908527 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.908377    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:21.908527 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.908395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" event={"ID":"1389556b-cd1b-4312-8ed0-798eb53093b4","Type":"ContainerStarted","Data":"2f63f12404083f1a4b96a9c942af1ccfa015ae8f1d6df9bdf6f7b33520903848"}
Jul 08 17:04:21.909893 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.909866    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 08 17:04:21.928241 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:21.928189    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podStartSLOduration=2.928172507 podStartE2EDuration="2.928172507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:04:21.927147891 +0000 UTC m=+1882.489125288" watchObservedRunningTime="2026-07-08 17:04:21.928172507 +0000 UTC m=+1882.490149891"
Jul 08 17:04:22.714198 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.714167    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 17:04:22.814859 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.814746    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:04:22.831301 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.831268    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/786486ea-9806-41d8-a8ca-8835eb3db625-proxy-tls\") pod \"786486ea-9806-41d8-a8ca-8835eb3db625\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") "
Jul 08 17:04:22.831460 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.831366    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/786486ea-9806-41d8-a8ca-8835eb3db625-kube-rbac-proxy-sar-config\") pod \"786486ea-9806-41d8-a8ca-8835eb3db625\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") "
Jul 08 17:04:22.831460 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.831436    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9t8j\" (UniqueName: \"kubernetes.io/projected/786486ea-9806-41d8-a8ca-8835eb3db625-kube-api-access-p9t8j\") pod \"786486ea-9806-41d8-a8ca-8835eb3db625\" (UID: \"786486ea-9806-41d8-a8ca-8835eb3db625\") "
Jul 08 17:04:22.831752 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.831713    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/786486ea-9806-41d8-a8ca-8835eb3db625-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "786486ea-9806-41d8-a8ca-8835eb3db625" (UID: "786486ea-9806-41d8-a8ca-8835eb3db625"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:04:22.833494 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.833461    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/786486ea-9806-41d8-a8ca-8835eb3db625-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "786486ea-9806-41d8-a8ca-8835eb3db625" (UID: "786486ea-9806-41d8-a8ca-8835eb3db625"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:04:22.833494 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.833478    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786486ea-9806-41d8-a8ca-8835eb3db625-kube-api-access-p9t8j" (OuterVolumeSpecName: "kube-api-access-p9t8j") pod "786486ea-9806-41d8-a8ca-8835eb3db625" (UID: "786486ea-9806-41d8-a8ca-8835eb3db625"). InnerVolumeSpecName "kube-api-access-p9t8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 17:04:22.914121 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.914089    2562 generic.go:356] "Generic (PLEG): container finished" podID="786486ea-9806-41d8-a8ca-8835eb3db625" containerID="075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7" exitCode=0
Jul 08 17:04:22.914311 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.914172    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"
Jul 08 17:04:22.914311 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.914177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" event={"ID":"786486ea-9806-41d8-a8ca-8835eb3db625","Type":"ContainerDied","Data":"075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7"}
Jul 08 17:04:22.914311 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.914252    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7" event={"ID":"786486ea-9806-41d8-a8ca-8835eb3db625","Type":"ContainerDied","Data":"84c0ecc807d9d8231bea249fb0b43191d218cb09d4d39db2a35009546b0fa563"}
Jul 08 17:04:22.914311 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.914288    2562 scope.go:117] "RemoveContainer" containerID="35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b"
Jul 08 17:04:22.914918 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.914888    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 08 17:04:22.923016 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.923002    2562 scope.go:117] "RemoveContainer" containerID="075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7"
Jul 08 17:04:22.931395 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.931379    2562 scope.go:117] "RemoveContainer" containerID="35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b"
Jul 08 17:04:22.931638 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:22.931620    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b\": container with ID starting with 35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b not found: ID does not exist" containerID="35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b"
Jul 08 17:04:22.931713 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.931651    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b"} err="failed to get container status \"35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b\": rpc error: code = NotFound desc = could not find container \"35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b\": container with ID starting with 35c8cc7ec9dc2479c8634693f5564545f39d3fb06dcf01431727556e2122fd4b not found: ID does not exist"
Jul 08 17:04:22.931713 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.931674    2562 scope.go:117] "RemoveContainer" containerID="075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7"
Jul 08 17:04:22.931946 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:22.931926    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7\": container with ID starting with 075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7 not found: ID does not exist" containerID="075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7"
Jul 08 17:04:22.932010 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.931951    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7"} err="failed to get container status \"075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7\": rpc error: code = NotFound desc = could not find container \"075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7\": container with ID starting with 075d284b4bd13e355858dce3628e74b78c454e6951daeba1cfe6f327ef2c04c7 not found: ID does not exist"
Jul 08 17:04:22.932076 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.932062    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/786486ea-9806-41d8-a8ca-8835eb3db625-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:04:22.932111 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.932079    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9t8j\" (UniqueName: \"kubernetes.io/projected/786486ea-9806-41d8-a8ca-8835eb3db625-kube-api-access-p9t8j\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:04:22.932111 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.932090    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/786486ea-9806-41d8-a8ca-8835eb3db625-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:04:22.935235 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.935211    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"]
Jul 08 17:04:22.939605 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:22.939586    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1f879-predictor-669f7c8b8b-qz6j7"]
Jul 08 17:04:24.029213 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:24.029179    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" path="/var/lib/kubelet/pods/786486ea-9806-41d8-a8ca-8835eb3db625/volumes"
Jul 08 17:04:27.813804 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:27.813767    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:04:27.919136 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:27.919108    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:04:27.919502 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:27.919477    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 08 17:04:32.814392 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:32.814353    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:04:32.814790 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:32.814485    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 17:04:37.814564 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:37.814529    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:04:37.919560 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:37.919518    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 08 17:04:42.814369 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:42.814327    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:04:47.814052 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:47.814015    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:04:47.920529 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:47.920488    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 08 17:04:49.461054 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.461028    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 17:04:49.650078 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.650047    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be147abb-e4e1-4eef-9ed1-1e4162a088ca-openshift-service-ca-bundle\") pod \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") "
Jul 08 17:04:49.650242 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.650084    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls\") pod \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\" (UID: \"be147abb-e4e1-4eef-9ed1-1e4162a088ca\") "
Jul 08 17:04:49.650405 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.650375    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be147abb-e4e1-4eef-9ed1-1e4162a088ca-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "be147abb-e4e1-4eef-9ed1-1e4162a088ca" (UID: "be147abb-e4e1-4eef-9ed1-1e4162a088ca"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:04:49.652013 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.651995    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "be147abb-e4e1-4eef-9ed1-1e4162a088ca" (UID: "be147abb-e4e1-4eef-9ed1-1e4162a088ca"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:04:49.750803 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.750770    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be147abb-e4e1-4eef-9ed1-1e4162a088ca-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:04:49.750803 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:49.750796    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be147abb-e4e1-4eef-9ed1-1e4162a088ca-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:04:50.039341 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.039313    2562 generic.go:356] "Generic (PLEG): container finished" podID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerID="ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07" exitCode=0
Jul 08 17:04:50.039461 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.039348    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"
Jul 08 17:04:50.039461 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.039369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" event={"ID":"be147abb-e4e1-4eef-9ed1-1e4162a088ca","Type":"ContainerDied","Data":"ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07"}
Jul 08 17:04:50.039461 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.039419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g" event={"ID":"be147abb-e4e1-4eef-9ed1-1e4162a088ca","Type":"ContainerDied","Data":"8e76e3a20e7a8f58f0a27186b4f576310978682e7971f5c8476eafe58679f8f3"}
Jul 08 17:04:50.039461 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.039442    2562 scope.go:117] "RemoveContainer" containerID="ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07"
Jul 08 17:04:50.047468 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.047451    2562 scope.go:117] "RemoveContainer" containerID="ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07"
Jul 08 17:04:50.047721 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:50.047704    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07\": container with ID starting with ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07 not found: ID does not exist" containerID="ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07"
Jul 08 17:04:50.047770 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.047732    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07"} err="failed to get container status \"ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07\": rpc error: code = NotFound desc = could not find container \"ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07\": container with ID starting with ec25442c29454f1419c49668517525240f6a08b0dd9ad37655c7f6a1720f3a07 not found: ID does not exist"
Jul 08 17:04:50.054482 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.054464    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"]
Jul 08 17:04:50.060579 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:50.060561    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1f879-76cbbb6865-l8c8g"]
Jul 08 17:04:52.029223 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:52.029188    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" path="/var/lib/kubelet/pods/be147abb-e4e1-4eef-9ed1-1e4162a088ca/volumes"
Jul 08 17:04:57.920002 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:57.919960    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 08 17:04:59.258390 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.258355    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"]
Jul 08 17:04:59.258759 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.258604    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" containerID="cri-o://fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c" gracePeriod=30
Jul 08 17:04:59.389087 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.389057    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"]
Jul 08 17:04:59.389312 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.389285    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container" containerID="cri-o://ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c" gracePeriod=30
Jul 08 17:04:59.389374 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.389312    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kube-rbac-proxy" containerID="cri-o://8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713" gracePeriod=30
Jul 08 17:04:59.467543 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467512    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"]
Jul 08 17:04:59.467862 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467837    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container"
Jul 08 17:04:59.467862 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467855    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467869    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467878    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467890    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kube-rbac-proxy"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467895    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kube-rbac-proxy"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467963    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kserve-container"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467975    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="786486ea-9806-41d8-a8ca-8835eb3db625" containerName="kube-rbac-proxy"
Jul 08 17:04:59.468018 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.467981    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="be147abb-e4e1-4eef-9ed1-1e4162a088ca" containerName="ensemble-graph-1f879"
Jul 08 17:04:59.472344 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.472327    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.474848 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.474807    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-73f94-kube-rbac-proxy-sar-config\""
Jul 08 17:04:59.474950 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.474914    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-73f94-predictor-serving-cert\""
Jul 08 17:04:59.482474 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.482455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"]
Jul 08 17:04:59.527129 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.527049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.527129 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.527094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gdpt\" (UniqueName: \"kubernetes.io/projected/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-api-access-5gdpt\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.527328 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.527174    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.630216 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.630179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.630417 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.630227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gdpt\" (UniqueName: \"kubernetes.io/projected/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-api-access-5gdpt\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.630417 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.630259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.630417 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:59.630374    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-73f94-predictor-serving-cert: secret "success-200-isvc-73f94-predictor-serving-cert" not found
Jul 08 17:04:59.630598 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:04:59.630466    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls podName:482ae0f4-77fe-41ca-b582-1cf28a9805f6 nodeName:}" failed. No retries permitted until 2026-07-08 17:05:00.130445089 +0000 UTC m=+1920.692422476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls") pod "success-200-isvc-73f94-predictor-68bc54bd54-h82px" (UID: "482ae0f4-77fe-41ca-b582-1cf28a9805f6") : secret "success-200-isvc-73f94-predictor-serving-cert" not found
Jul 08 17:04:59.631004 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.630984    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:04:59.639477 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:04:59.639451    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gdpt\" (UniqueName: \"kubernetes.io/projected/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-api-access-5gdpt\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:00.081004 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:00.080970    2562 generic.go:356] "Generic (PLEG): container finished" podID="ee220760-6390-446a-b641-f2bfce2f1c52" containerID="8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713" exitCode=2
Jul 08 17:05:00.081220 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:00.081041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" event={"ID":"ee220760-6390-446a-b641-f2bfce2f1c52","Type":"ContainerDied","Data":"8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713"}
Jul 08 17:05:00.134148 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:00.134118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:00.136661 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:00.136638    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls\") pod \"success-200-isvc-73f94-predictor-68bc54bd54-h82px\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") " pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:00.383022 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:00.382934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:00.517444 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:00.517420    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"]
Jul 08 17:05:00.519771 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:05:00.519743    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482ae0f4_77fe_41ca_b582_1cf28a9805f6.slice/crio-fc27166fa0b5af3bfb7be7d624837fe6c0345be6bf26b68c90fcb1cf3c06f4f8 WatchSource:0}: Error finding container fc27166fa0b5af3bfb7be7d624837fe6c0345be6bf26b68c90fcb1cf3c06f4f8: Status 404 returned error can't find the container with id fc27166fa0b5af3bfb7be7d624837fe6c0345be6bf26b68c90fcb1cf3c06f4f8
Jul 08 17:05:01.087043 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:01.087012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" event={"ID":"482ae0f4-77fe-41ca-b582-1cf28a9805f6","Type":"ContainerStarted","Data":"15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419"}
Jul 08 17:05:01.087233 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:01.087078    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" event={"ID":"482ae0f4-77fe-41ca-b582-1cf28a9805f6","Type":"ContainerStarted","Data":"f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda"}
Jul 08 17:05:01.087233 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:01.087098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" event={"ID":"482ae0f4-77fe-41ca-b582-1cf28a9805f6","Type":"ContainerStarted","Data":"fc27166fa0b5af3bfb7be7d624837fe6c0345be6bf26b68c90fcb1cf3c06f4f8"}
Jul 08 17:05:01.087233 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:01.087149    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:01.108352 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:01.108307    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podStartSLOduration=2.108292692 podStartE2EDuration="2.108292692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:04:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:05:01.106050735 +0000 UTC m=+1921.668028119" watchObservedRunningTime="2026-07-08 17:05:01.108292692 +0000 UTC m=+1921.670270076"
Jul 08 17:05:01.986606 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:01.986571    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:02.090968 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.090928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:02.092377 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.092351    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 08 17:05:02.631345 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.631318    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 17:05:02.655335 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.655313    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee220760-6390-446a-b641-f2bfce2f1c52-proxy-tls\") pod \"ee220760-6390-446a-b641-f2bfce2f1c52\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") "
Jul 08 17:05:02.655475 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.655385    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee220760-6390-446a-b641-f2bfce2f1c52-kube-rbac-proxy-sar-config\") pod \"ee220760-6390-446a-b641-f2bfce2f1c52\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") "
Jul 08 17:05:02.655475 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.655419    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-626dp\" (UniqueName: \"kubernetes.io/projected/ee220760-6390-446a-b641-f2bfce2f1c52-kube-api-access-626dp\") pod \"ee220760-6390-446a-b641-f2bfce2f1c52\" (UID: \"ee220760-6390-446a-b641-f2bfce2f1c52\") "
Jul 08 17:05:02.655806 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.655766    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee220760-6390-446a-b641-f2bfce2f1c52-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ee220760-6390-446a-b641-f2bfce2f1c52" (UID: "ee220760-6390-446a-b641-f2bfce2f1c52"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:05:02.657718 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.657684    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee220760-6390-446a-b641-f2bfce2f1c52-kube-api-access-626dp" (OuterVolumeSpecName: "kube-api-access-626dp") pod "ee220760-6390-446a-b641-f2bfce2f1c52" (UID: "ee220760-6390-446a-b641-f2bfce2f1c52"). InnerVolumeSpecName "kube-api-access-626dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 17:05:02.657861 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.657760    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee220760-6390-446a-b641-f2bfce2f1c52-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ee220760-6390-446a-b641-f2bfce2f1c52" (UID: "ee220760-6390-446a-b641-f2bfce2f1c52"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:05:02.756100 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.756016    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee220760-6390-446a-b641-f2bfce2f1c52-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:02.756100 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.756047    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee220760-6390-446a-b641-f2bfce2f1c52-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:02.756100 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:02.756057    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-626dp\" (UniqueName: \"kubernetes.io/projected/ee220760-6390-446a-b641-f2bfce2f1c52-kube-api-access-626dp\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:03.097102 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.097064    2562 generic.go:356] "Generic (PLEG): container finished" podID="ee220760-6390-446a-b641-f2bfce2f1c52" containerID="ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c" exitCode=0
Jul 08 17:05:03.097503 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.097126    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"
Jul 08 17:05:03.097503 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.097144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" event={"ID":"ee220760-6390-446a-b641-f2bfce2f1c52","Type":"ContainerDied","Data":"ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c"}
Jul 08 17:05:03.097503 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.097200    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4" event={"ID":"ee220760-6390-446a-b641-f2bfce2f1c52","Type":"ContainerDied","Data":"db4dd33afd80185723bc36e0d1cf2d42d77dbae9bdd79851cc5611c553b1e047"}
Jul 08 17:05:03.097503 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.097229    2562 scope.go:117] "RemoveContainer" containerID="8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713"
Jul 08 17:05:03.097714 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.097630    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 08 17:05:03.105944 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.105927    2562 scope.go:117] "RemoveContainer" containerID="ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c"
Jul 08 17:05:03.113386 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.113370    2562 scope.go:117] "RemoveContainer" containerID="8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713"
Jul 08 17:05:03.113624 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:05:03.113605    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713\": container with ID starting with 8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713 not found: ID does not exist" containerID="8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713"
Jul 08 17:05:03.113707 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.113637    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713"} err="failed to get container status \"8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713\": rpc error: code = NotFound desc = could not find container \"8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713\": container with ID starting with 8f29bbe96f96f83fde6151ab105c26310cac3d998935724cf7e2f858a4b7b713 not found: ID does not exist"
Jul 08 17:05:03.113707 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.113661    2562 scope.go:117] "RemoveContainer" containerID="ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c"
Jul 08 17:05:03.113944 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:05:03.113926    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c\": container with ID starting with ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c not found: ID does not exist" containerID="ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c"
Jul 08 17:05:03.113989 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.113951    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c"} err="failed to get container status \"ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c\": rpc error: code = NotFound desc = could not find container \"ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c\": container with ID starting with ebb8a6a841c9cbf51be304894ea003e56a6dd0d286474b91833f247fb77a6e2c not found: ID does not exist"
Jul 08 17:05:03.124063 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.124042    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"]
Jul 08 17:05:03.127178 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:03.127160    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f6dd6-predictor-6df86f94d4-vk2v4"]
Jul 08 17:05:04.029488 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:04.029456    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" path="/var/lib/kubelet/pods/ee220760-6390-446a-b641-f2bfce2f1c52/volumes"
Jul 08 17:05:06.987176 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:06.987134    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:07.920494 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:07.920467    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:05:08.102083 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:08.102058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:08.102530 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:08.102503    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 08 17:05:11.986663 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:11.986625    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:11.987090 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:11.986746    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 17:05:16.987024 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:16.986982    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:18.103394 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:18.103354    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 08 17:05:19.560543 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560510    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"]
Jul 08 17:05:19.560985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560818    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kube-rbac-proxy"
Jul 08 17:05:19.560985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560854    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kube-rbac-proxy"
Jul 08 17:05:19.560985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560888    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container"
Jul 08 17:05:19.560985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560894    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container"
Jul 08 17:05:19.560985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560957    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kserve-container"
Jul 08 17:05:19.560985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.560967    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee220760-6390-446a-b641-f2bfce2f1c52" containerName="kube-rbac-proxy"
Jul 08 17:05:19.565198 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.565166    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.568607 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.568580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-78c4a-serving-cert\""
Jul 08 17:05:19.568745 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.568634    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-78c4a-kube-rbac-proxy-sar-config\""
Jul 08 17:05:19.575328 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.575309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"]
Jul 08 17:05:19.688588 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.688552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f515122-b53d-41f8-aea1-ed4c75c91d74-proxy-tls\") pod \"splitter-graph-78c4a-56cf4ffc7b-l8qvw\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") " pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.688588 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.688592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f515122-b53d-41f8-aea1-ed4c75c91d74-openshift-service-ca-bundle\") pod \"splitter-graph-78c4a-56cf4ffc7b-l8qvw\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") " pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.789898 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.789845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f515122-b53d-41f8-aea1-ed4c75c91d74-proxy-tls\") pod \"splitter-graph-78c4a-56cf4ffc7b-l8qvw\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") " pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.789898 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.789905    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f515122-b53d-41f8-aea1-ed4c75c91d74-openshift-service-ca-bundle\") pod \"splitter-graph-78c4a-56cf4ffc7b-l8qvw\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") " pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.790537 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.790519    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f515122-b53d-41f8-aea1-ed4c75c91d74-openshift-service-ca-bundle\") pod \"splitter-graph-78c4a-56cf4ffc7b-l8qvw\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") " pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.792202 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.792171    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f515122-b53d-41f8-aea1-ed4c75c91d74-proxy-tls\") pod \"splitter-graph-78c4a-56cf4ffc7b-l8qvw\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") " pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:19.877310 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:19.877270    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:20.005603 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:20.005562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"]
Jul 08 17:05:20.009281 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:05:20.009250    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f515122_b53d_41f8_aea1_ed4c75c91d74.slice/crio-be524f69b8eef51604851e00224dfb080b03570a62473704853562f5570644e7 WatchSource:0}: Error finding container be524f69b8eef51604851e00224dfb080b03570a62473704853562f5570644e7: Status 404 returned error can't find the container with id be524f69b8eef51604851e00224dfb080b03570a62473704853562f5570644e7
Jul 08 17:05:20.172544 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:20.172454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" event={"ID":"3f515122-b53d-41f8-aea1-ed4c75c91d74","Type":"ContainerStarted","Data":"a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb"}
Jul 08 17:05:20.172544 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:20.172513    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:20.172544 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:20.172532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" event={"ID":"3f515122-b53d-41f8-aea1-ed4c75c91d74","Type":"ContainerStarted","Data":"be524f69b8eef51604851e00224dfb080b03570a62473704853562f5570644e7"}
Jul 08 17:05:20.190510 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:20.190469    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podStartSLOduration=1.190454917 podStartE2EDuration="1.190454917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:05:20.18898077 +0000 UTC m=+1940.750958149" watchObservedRunningTime="2026-07-08 17:05:20.190454917 +0000 UTC m=+1940.752432306"
Jul 08 17:05:21.986738 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:21.986695    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:26.182908 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:26.182880    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:26.986630 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:26.986589    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:28.103354 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:28.103310    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 08 17:05:29.648417 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.648378    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"]
Jul 08 17:05:29.648860 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.648610    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" containerID="cri-o://a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb" gracePeriod=30
Jul 08 17:05:29.754127 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.754087    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"]
Jul 08 17:05:29.754329 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.754307    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container" containerID="cri-o://92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161" gracePeriod=30
Jul 08 17:05:29.754397 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.754349    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kube-rbac-proxy" containerID="cri-o://24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a" gracePeriod=30
Jul 08 17:05:29.836594 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.836563    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"]
Jul 08 17:05:29.840019 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.839991    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:29.842366 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.842278    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-91fdc-predictor-serving-cert\""
Jul 08 17:05:29.842496 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.842479    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-91fdc-kube-rbac-proxy-sar-config\""
Jul 08 17:05:29.852318 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.852193    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"]
Jul 08 17:05:29.920068 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.920047    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 17:05:29.977601 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.977570    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c532c61d-3d07-47d9-bad2-19801079abab-openshift-service-ca-bundle\") pod \"c532c61d-3d07-47d9-bad2-19801079abab\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") "
Jul 08 17:05:29.977756 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.977631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls\") pod \"c532c61d-3d07-47d9-bad2-19801079abab\" (UID: \"c532c61d-3d07-47d9-bad2-19801079abab\") "
Jul 08 17:05:29.977806 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.977785    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qtfv\" (UniqueName: \"kubernetes.io/projected/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-api-access-2qtfv\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:29.977870 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.977841    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-proxy-tls\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:29.977945 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.977929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:29.978010 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.977988    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c532c61d-3d07-47d9-bad2-19801079abab-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c532c61d-3d07-47d9-bad2-19801079abab" (UID: "c532c61d-3d07-47d9-bad2-19801079abab"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:05:29.979665 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:29.979643    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c532c61d-3d07-47d9-bad2-19801079abab" (UID: "c532c61d-3d07-47d9-bad2-19801079abab"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:05:30.079146 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.079111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-proxy-tls\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.079306 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.079196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.079306 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.079259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qtfv\" (UniqueName: \"kubernetes.io/projected/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-api-access-2qtfv\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.079306 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.079299    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c532c61d-3d07-47d9-bad2-19801079abab-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:30.079448 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.079314    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c532c61d-3d07-47d9-bad2-19801079abab-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:30.079882 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.079862    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.081462 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.081436    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-proxy-tls\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.087223 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.087202    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qtfv\" (UniqueName: \"kubernetes.io/projected/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-api-access-2qtfv\") pod \"success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") " pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.152955 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.152883    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:30.216942 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.216905    2562 generic.go:356] "Generic (PLEG): container finished" podID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerID="24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a" exitCode=2
Jul 08 17:05:30.217078 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.216972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" event={"ID":"1389556b-cd1b-4312-8ed0-798eb53093b4","Type":"ContainerDied","Data":"24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a"}
Jul 08 17:05:30.218285 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.218261    2562 generic.go:356] "Generic (PLEG): container finished" podID="c532c61d-3d07-47d9-bad2-19801079abab" containerID="fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c" exitCode=0
Jul 08 17:05:30.218390 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.218313    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"
Jul 08 17:05:30.218390 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.218321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" event={"ID":"c532c61d-3d07-47d9-bad2-19801079abab","Type":"ContainerDied","Data":"fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c"}
Jul 08 17:05:30.218390 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.218355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf" event={"ID":"c532c61d-3d07-47d9-bad2-19801079abab","Type":"ContainerDied","Data":"a5ff3755e56cd752c3c6e5a837a876a8d052aadbe5898c96ccbefd50da1549f0"}
Jul 08 17:05:30.218390 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.218376    2562 scope.go:117] "RemoveContainer" containerID="fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c"
Jul 08 17:05:30.227242 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.227222    2562 scope.go:117] "RemoveContainer" containerID="fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c"
Jul 08 17:05:30.227481 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:05:30.227465    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c\": container with ID starting with fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c not found: ID does not exist" containerID="fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c"
Jul 08 17:05:30.227549 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.227489    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c"} err="failed to get container status \"fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c\": rpc error: code = NotFound desc = could not find container \"fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c\": container with ID starting with fac2f62a1b057390b8117fc5afa7bb209efe4350b82b45f59df83c954626b29c not found: ID does not exist"
Jul 08 17:05:30.234749 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.234712    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"]
Jul 08 17:05:30.236534 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.236512    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f6dd6-cbdd594c5-jgpqf"]
Jul 08 17:05:30.288368 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:30.288344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"]
Jul 08 17:05:30.290591 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:05:30.290557    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb571fa_5d32_4a82_a0c2_34d9c659a80e.slice/crio-48d38a516b2d5b1a4c0e65158137319ee014e472f295a3db8ee73b656bffc37d WatchSource:0}: Error finding container 48d38a516b2d5b1a4c0e65158137319ee014e472f295a3db8ee73b656bffc37d: Status 404 returned error can't find the container with id 48d38a516b2d5b1a4c0e65158137319ee014e472f295a3db8ee73b656bffc37d
Jul 08 17:05:31.181105 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:31.181053    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:31.225039 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:31.224998    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" event={"ID":"2eb571fa-5d32-4a82-a0c2-34d9c659a80e","Type":"ContainerStarted","Data":"1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7"}
Jul 08 17:05:31.225224 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:31.225116    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:31.225224 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:31.225151    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" event={"ID":"2eb571fa-5d32-4a82-a0c2-34d9c659a80e","Type":"ContainerStarted","Data":"14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031"}
Jul 08 17:05:31.225224 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:31.225181    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" event={"ID":"2eb571fa-5d32-4a82-a0c2-34d9c659a80e","Type":"ContainerStarted","Data":"48d38a516b2d5b1a4c0e65158137319ee014e472f295a3db8ee73b656bffc37d"}
Jul 08 17:05:31.244382 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:31.244330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podStartSLOduration=2.244313371 podStartE2EDuration="2.244313371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:05:31.243014502 +0000 UTC m=+1951.804991886" watchObservedRunningTime="2026-07-08 17:05:31.244313371 +0000 UTC m=+1951.806290756"
Jul 08 17:05:32.030872 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:32.030814    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c532c61d-3d07-47d9-bad2-19801079abab" path="/var/lib/kubelet/pods/c532c61d-3d07-47d9-bad2-19801079abab/volumes"
Jul 08 17:05:32.230612 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:32.230576    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:32.231927 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:32.231897    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 08 17:05:32.918587 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:32.918562    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:05:33.005371 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.005289    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") pod \"1389556b-cd1b-4312-8ed0-798eb53093b4\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") "
Jul 08 17:05:33.005528 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.005383    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-rbac-proxy-sar-config\") pod \"1389556b-cd1b-4312-8ed0-798eb53093b4\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") "
Jul 08 17:05:33.005528 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.005402    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gkvg\" (UniqueName: \"kubernetes.io/projected/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-api-access-2gkvg\") pod \"1389556b-cd1b-4312-8ed0-798eb53093b4\" (UID: \"1389556b-cd1b-4312-8ed0-798eb53093b4\") "
Jul 08 17:05:33.005765 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.005742    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1389556b-cd1b-4312-8ed0-798eb53093b4" (UID: "1389556b-cd1b-4312-8ed0-798eb53093b4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:05:33.007348 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.007322    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1389556b-cd1b-4312-8ed0-798eb53093b4" (UID: "1389556b-cd1b-4312-8ed0-798eb53093b4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:05:33.007441 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.007353    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-api-access-2gkvg" (OuterVolumeSpecName: "kube-api-access-2gkvg") pod "1389556b-cd1b-4312-8ed0-798eb53093b4" (UID: "1389556b-cd1b-4312-8ed0-798eb53093b4"). InnerVolumeSpecName "kube-api-access-2gkvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 17:05:33.106600 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.106565    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:33.106600 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.106594    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gkvg\" (UniqueName: \"kubernetes.io/projected/1389556b-cd1b-4312-8ed0-798eb53093b4-kube-api-access-2gkvg\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:33.106600 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.106607    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1389556b-cd1b-4312-8ed0-798eb53093b4-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:05:33.237222 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.237184    2562 generic.go:356] "Generic (PLEG): container finished" podID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerID="92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161" exitCode=0
Jul 08 17:05:33.237680 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.237272    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"
Jul 08 17:05:33.237680 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.237271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" event={"ID":"1389556b-cd1b-4312-8ed0-798eb53093b4","Type":"ContainerDied","Data":"92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161"}
Jul 08 17:05:33.237680 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.237439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" event={"ID":"1389556b-cd1b-4312-8ed0-798eb53093b4","Type":"ContainerDied","Data":"2f63f12404083f1a4b96a9c942af1ccfa015ae8f1d6df9bdf6f7b33520903848"}
Jul 08 17:05:33.237680 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.237486    2562 scope.go:117] "RemoveContainer" containerID="24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a"
Jul 08 17:05:33.237680 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.237534    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 08 17:05:33.246257 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.246242    2562 scope.go:117] "RemoveContainer" containerID="92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161"
Jul 08 17:05:33.253665 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.253647    2562 scope.go:117] "RemoveContainer" containerID="24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a"
Jul 08 17:05:33.253942 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:05:33.253923    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a\": container with ID starting with 24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a not found: ID does not exist" containerID="24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a"
Jul 08 17:05:33.253997 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.253951    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a"} err="failed to get container status \"24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a\": rpc error: code = NotFound desc = could not find container \"24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a\": container with ID starting with 24a8531627e837e8acc4fd4dafb2aa3dae55c1e90a38d2926cbf3d6502d54e5a not found: ID does not exist"
Jul 08 17:05:33.253997 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.253969    2562 scope.go:117] "RemoveContainer" containerID="92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161"
Jul 08 17:05:33.254217 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:05:33.254200    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161\": container with ID starting with 92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161 not found: ID does not exist" containerID="92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161"
Jul 08 17:05:33.254274 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.254226    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161"} err="failed to get container status \"92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161\": rpc error: code = NotFound desc = could not find container \"92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161\": container with ID starting with 92804d6ff8cb4f05f2fe407f3658da6355d7d3a3d06dd4846d09801ffc1e6161 not found: ID does not exist"
Jul 08 17:05:33.258443 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.258353    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"]
Jul 08 17:05:33.259785 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.259763    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6"]
Jul 08 17:05:33.915103 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:33.915046    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78c4a-predictor-7cc8c8bf4b-mxbm6" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": context deadline exceeded"
Jul 08 17:05:34.031046 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:34.030999    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" path="/var/lib/kubelet/pods/1389556b-cd1b-4312-8ed0-798eb53093b4/volumes"
Jul 08 17:05:36.181273 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:36.181230    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:38.102665 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:38.102630    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 08 17:05:38.242061 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:38.242035    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:05:38.242486 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:38.242452    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 08 17:05:41.181154 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:41.181114    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:41.181528 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:41.181220    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:05:46.181664 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:46.181627    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:48.103797 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:48.103768    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:05:48.242430 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:48.242395    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 08 17:05:51.180594 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:51.180561    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:56.180694 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:56.180606    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:05:58.243050 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:58.243015    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 08 17:05:59.537765 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.537376    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"]
Jul 08 17:05:59.538280 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538252    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6"
Jul 08 17:05:59.538345 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538284    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6"
Jul 08 17:05:59.538345 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538318    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kube-rbac-proxy"
Jul 08 17:05:59.538345 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538326    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kube-rbac-proxy"
Jul 08 17:05:59.538345 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538339    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container"
Jul 08 17:05:59.538534 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538351    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container"
Jul 08 17:05:59.538584 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538544    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kube-rbac-proxy"
Jul 08 17:05:59.538584 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538562    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1389556b-cd1b-4312-8ed0-798eb53093b4" containerName="kserve-container"
Jul 08 17:05:59.538584 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.538581    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c532c61d-3d07-47d9-bad2-19801079abab" containerName="sequence-graph-f6dd6"
Jul 08 17:05:59.544625 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.544604    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.547111 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.547078    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-73f94-kube-rbac-proxy-sar-config\""
Jul 08 17:05:59.547248 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.547131    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-73f94-serving-cert\""
Jul 08 17:05:59.552233 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.552201    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"]
Jul 08 17:05:59.618594 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.618552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4bee131-c4b3-45cf-a941-249bf8624a7a-proxy-tls\") pod \"switch-graph-73f94-65b6995f6-nlths\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") " pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.618864 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.618614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4bee131-c4b3-45cf-a941-249bf8624a7a-openshift-service-ca-bundle\") pod \"switch-graph-73f94-65b6995f6-nlths\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") " pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.719991 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.719964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4bee131-c4b3-45cf-a941-249bf8624a7a-openshift-service-ca-bundle\") pod \"switch-graph-73f94-65b6995f6-nlths\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") " pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.720103 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.720020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4bee131-c4b3-45cf-a941-249bf8624a7a-proxy-tls\") pod \"switch-graph-73f94-65b6995f6-nlths\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") " pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.720506 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.720485    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4bee131-c4b3-45cf-a941-249bf8624a7a-openshift-service-ca-bundle\") pod \"switch-graph-73f94-65b6995f6-nlths\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") " pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.722151 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.722132    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4bee131-c4b3-45cf-a941-249bf8624a7a-proxy-tls\") pod \"switch-graph-73f94-65b6995f6-nlths\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") " pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.856731 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.856700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:05:59.989091 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:05:59.989069    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"]
Jul 08 17:05:59.991160 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:05:59.991124    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4bee131_c4b3_45cf_a941_249bf8624a7a.slice/crio-2a375144126c18ea0af508a90e0de429554256d7ff1e80ddf9097124622d8c2d WatchSource:0}: Error finding container 2a375144126c18ea0af508a90e0de429554256d7ff1e80ddf9097124622d8c2d: Status 404 returned error can't find the container with id 2a375144126c18ea0af508a90e0de429554256d7ff1e80ddf9097124622d8c2d
Jul 08 17:06:00.280835 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.280798    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:06:00.325021 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.324998    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f515122-b53d-41f8-aea1-ed4c75c91d74-proxy-tls\") pod \"3f515122-b53d-41f8-aea1-ed4c75c91d74\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") "
Jul 08 17:06:00.325161 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.325081    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f515122-b53d-41f8-aea1-ed4c75c91d74-openshift-service-ca-bundle\") pod \"3f515122-b53d-41f8-aea1-ed4c75c91d74\" (UID: \"3f515122-b53d-41f8-aea1-ed4c75c91d74\") "
Jul 08 17:06:00.325432 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.325411    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f515122-b53d-41f8-aea1-ed4c75c91d74-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3f515122-b53d-41f8-aea1-ed4c75c91d74" (UID: "3f515122-b53d-41f8-aea1-ed4c75c91d74"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:06:00.327062 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.327045    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f515122-b53d-41f8-aea1-ed4c75c91d74-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3f515122-b53d-41f8-aea1-ed4c75c91d74" (UID: "3f515122-b53d-41f8-aea1-ed4c75c91d74"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:06:00.352867 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.352836    2562 generic.go:356] "Generic (PLEG): container finished" podID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerID="a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb" exitCode=0
Jul 08 17:06:00.353037 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.352869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" event={"ID":"3f515122-b53d-41f8-aea1-ed4c75c91d74","Type":"ContainerDied","Data":"a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb"}
Jul 08 17:06:00.353037 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.352893    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"
Jul 08 17:06:00.353037 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.352933    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw" event={"ID":"3f515122-b53d-41f8-aea1-ed4c75c91d74","Type":"ContainerDied","Data":"be524f69b8eef51604851e00224dfb080b03570a62473704853562f5570644e7"}
Jul 08 17:06:00.353037 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.352964    2562 scope.go:117] "RemoveContainer" containerID="a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb"
Jul 08 17:06:00.354437 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.354414    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" event={"ID":"b4bee131-c4b3-45cf-a941-249bf8624a7a","Type":"ContainerStarted","Data":"bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0"}
Jul 08 17:06:00.354534 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.354464    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" event={"ID":"b4bee131-c4b3-45cf-a941-249bf8624a7a","Type":"ContainerStarted","Data":"2a375144126c18ea0af508a90e0de429554256d7ff1e80ddf9097124622d8c2d"}
Jul 08 17:06:00.354534 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.354519    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:06:00.361409 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.361390    2562 scope.go:117] "RemoveContainer" containerID="a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb"
Jul 08 17:06:00.361643 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:06:00.361624    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb\": container with ID starting with a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb not found: ID does not exist" containerID="a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb"
Jul 08 17:06:00.361681 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.361657    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb"} err="failed to get container status \"a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb\": rpc error: code = NotFound desc = could not find container \"a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb\": container with ID starting with a56f6d81b3cd5ec85e9d84e27a261e4f8ed0d8f72d7ccb88d07988732d9b53bb not found: ID does not exist"
Jul 08 17:06:00.374137 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.374094    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podStartSLOduration=1.374081892 podStartE2EDuration="1.374081892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:06:00.372481351 +0000 UTC m=+1980.934458732" watchObservedRunningTime="2026-07-08 17:06:00.374081892 +0000 UTC m=+1980.936059311"
Jul 08 17:06:00.387475 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.387428    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"]
Jul 08 17:06:00.390816 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.390798    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78c4a-56cf4ffc7b-l8qvw"]
Jul 08 17:06:00.426397 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.426379    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f515122-b53d-41f8-aea1-ed4c75c91d74-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:06:00.426397 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:00.426397    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f515122-b53d-41f8-aea1-ed4c75c91d74-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:06:02.029164 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:02.029132    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" path="/var/lib/kubelet/pods/3f515122-b53d-41f8-aea1-ed4c75c91d74/volumes"
Jul 08 17:06:06.365596 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:06.365565    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:06:08.242873 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:08.242802    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 08 17:06:18.243698 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:18.243669    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:06:29.904334 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.904299    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"]
Jul 08 17:06:29.904742 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.904625    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a"
Jul 08 17:06:29.904742 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.904635    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a"
Jul 08 17:06:29.904742 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.904683    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f515122-b53d-41f8-aea1-ed4c75c91d74" containerName="splitter-graph-78c4a"
Jul 08 17:06:29.907650 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.907632    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:29.910333 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.910308    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-91fdc-serving-cert\""
Jul 08 17:06:29.910586 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.910571    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-91fdc-kube-rbac-proxy-sar-config\""
Jul 08 17:06:29.917130 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.917106    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"]
Jul 08 17:06:29.958705 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.958670    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e352b067-8f1d-4604-b3ad-4731c9c07ef5-openshift-service-ca-bundle\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:29.958883 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:29.958793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.059507 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.059476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e352b067-8f1d-4604-b3ad-4731c9c07ef5-openshift-service-ca-bundle\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.059715 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.059575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.059715 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:06:30.059685    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-91fdc-serving-cert: secret "splitter-graph-91fdc-serving-cert" not found
Jul 08 17:06:30.059868 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:06:30.059740    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls podName:e352b067-8f1d-4604-b3ad-4731c9c07ef5 nodeName:}" failed. No retries permitted until 2026-07-08 17:06:30.559722512 +0000 UTC m=+2011.121699874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls") pod "splitter-graph-91fdc-d7f8d8f4c-q42gc" (UID: "e352b067-8f1d-4604-b3ad-4731c9c07ef5") : secret "splitter-graph-91fdc-serving-cert" not found
Jul 08 17:06:30.060221 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.060196    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e352b067-8f1d-4604-b3ad-4731c9c07ef5-openshift-service-ca-bundle\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.563888 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.563857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.566212 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.566192    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls\") pod \"splitter-graph-91fdc-d7f8d8f4c-q42gc\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") " pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.819137 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.819048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:30.954767 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:30.954740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"]
Jul 08 17:06:30.956425 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:06:30.956400    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode352b067_8f1d_4604_b3ad_4731c9c07ef5.slice/crio-337c4b92b28a55fe7a545ab67af7372196283c927925056f1b824f90b7b71c67 WatchSource:0}: Error finding container 337c4b92b28a55fe7a545ab67af7372196283c927925056f1b824f90b7b71c67: Status 404 returned error can't find the container with id 337c4b92b28a55fe7a545ab67af7372196283c927925056f1b824f90b7b71c67
Jul 08 17:06:31.480476 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:31.480438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" event={"ID":"e352b067-8f1d-4604-b3ad-4731c9c07ef5","Type":"ContainerStarted","Data":"5e63b621297f4d5fe18f2254bdd5d5b71e27f866998ea3fe564645a8feb231f1"}
Jul 08 17:06:31.480660 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:31.480495    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" event={"ID":"e352b067-8f1d-4604-b3ad-4731c9c07ef5","Type":"ContainerStarted","Data":"337c4b92b28a55fe7a545ab67af7372196283c927925056f1b824f90b7b71c67"}
Jul 08 17:06:31.480660 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:31.480523    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:37.489315 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:37.489280    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:06:37.504655 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:06:37.504598    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podStartSLOduration=8.504579125 podStartE2EDuration="8.504579125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:06:31.502266344 +0000 UTC m=+2012.064243729" watchObservedRunningTime="2026-07-08 17:06:37.504579125 +0000 UTC m=+2018.066556510"
Jul 08 17:08:03.398414 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:08:03.398383    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:08:03.401792 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:08:03.401763    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:13:03.422260 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:13:03.422230    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:13:03.425833 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:13:03.425799    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:14:44.536919 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:44.536881    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"]
Jul 08 17:14:44.537550 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:44.537077    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" containerID="cri-o://5e63b621297f4d5fe18f2254bdd5d5b71e27f866998ea3fe564645a8feb231f1" gracePeriod=30
Jul 08 17:14:44.661602 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:44.661574    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"]
Jul 08 17:14:44.661871 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:44.661800    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container" containerID="cri-o://14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031" gracePeriod=30
Jul 08 17:14:44.661871 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:44.661844    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kube-rbac-proxy" containerID="cri-o://1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7" gracePeriod=30
Jul 08 17:14:45.519646 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:45.519609    2562 generic.go:356] "Generic (PLEG): container finished" podID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerID="1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7" exitCode=2
Jul 08 17:14:45.519852 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:45.519685    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" event={"ID":"2eb571fa-5d32-4a82-a0c2-34d9c659a80e","Type":"ContainerDied","Data":"1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7"}
Jul 08 17:14:47.488724 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.488688    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:14:47.707388 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.707368    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:14:47.793335 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.793244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qtfv\" (UniqueName: \"kubernetes.io/projected/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-api-access-2qtfv\") pod \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") "
Jul 08 17:14:47.793497 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.793349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-proxy-tls\") pod \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") "
Jul 08 17:14:47.793497 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.793386    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-rbac-proxy-sar-config\") pod \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\" (UID: \"2eb571fa-5d32-4a82-a0c2-34d9c659a80e\") "
Jul 08 17:14:47.793788 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.793765    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2eb571fa-5d32-4a82-a0c2-34d9c659a80e" (UID: "2eb571fa-5d32-4a82-a0c2-34d9c659a80e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:14:47.795432 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.795409    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2eb571fa-5d32-4a82-a0c2-34d9c659a80e" (UID: "2eb571fa-5d32-4a82-a0c2-34d9c659a80e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:14:47.795531 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.795493    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-api-access-2qtfv" (OuterVolumeSpecName: "kube-api-access-2qtfv") pod "2eb571fa-5d32-4a82-a0c2-34d9c659a80e" (UID: "2eb571fa-5d32-4a82-a0c2-34d9c659a80e"). InnerVolumeSpecName "kube-api-access-2qtfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 17:14:47.894871 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.894815    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:14:47.894871 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.894869    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:14:47.895050 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:47.894880    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qtfv\" (UniqueName: \"kubernetes.io/projected/2eb571fa-5d32-4a82-a0c2-34d9c659a80e-kube-api-access-2qtfv\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:14:48.533005 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.532970    2562 generic.go:356] "Generic (PLEG): container finished" podID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerID="14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031" exitCode=0
Jul 08 17:14:48.533410 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.533017    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"
Jul 08 17:14:48.533410 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.533063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" event={"ID":"2eb571fa-5d32-4a82-a0c2-34d9c659a80e","Type":"ContainerDied","Data":"14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031"}
Jul 08 17:14:48.533410 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.533134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9" event={"ID":"2eb571fa-5d32-4a82-a0c2-34d9c659a80e","Type":"ContainerDied","Data":"48d38a516b2d5b1a4c0e65158137319ee014e472f295a3db8ee73b656bffc37d"}
Jul 08 17:14:48.533410 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.533171    2562 scope.go:117] "RemoveContainer" containerID="1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7"
Jul 08 17:14:48.541460 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.541442    2562 scope.go:117] "RemoveContainer" containerID="14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031"
Jul 08 17:14:48.548786 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.548768    2562 scope.go:117] "RemoveContainer" containerID="1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7"
Jul 08 17:14:48.549083 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:14:48.549064    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7\": container with ID starting with 1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7 not found: ID does not exist" containerID="1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7"
Jul 08 17:14:48.549138 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.549093    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7"} err="failed to get container status \"1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7\": rpc error: code = NotFound desc = could not find container \"1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7\": container with ID starting with 1b55baf490d9e852fe7bf838869cc2c12e1b1fbf050e42897f7cb0661d0dd3d7 not found: ID does not exist"
Jul 08 17:14:48.549138 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.549109    2562 scope.go:117] "RemoveContainer" containerID="14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031"
Jul 08 17:14:48.549346 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:14:48.549328    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031\": container with ID starting with 14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031 not found: ID does not exist" containerID="14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031"
Jul 08 17:14:48.549411 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.549354    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031"} err="failed to get container status \"14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031\": rpc error: code = NotFound desc = could not find container \"14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031\": container with ID starting with 14650b7976f4891b21e753c5acdae5127ea141cb4112a625b07d9fb4b515e031 not found: ID does not exist"
Jul 08 17:14:48.549812 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.549794    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"]
Jul 08 17:14:48.553548 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:48.553529    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-91fdc-predictor-768bfc64c4-lvnl9"]
Jul 08 17:14:50.028189 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:50.028158    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" path="/var/lib/kubelet/pods/2eb571fa-5d32-4a82-a0c2-34d9c659a80e/volumes"
Jul 08 17:14:52.488400 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:52.488312    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:14:57.487893 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:57.487852    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:14:57.488302 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:14:57.487975    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:15:02.488814 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:02.488772    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:15:07.488492 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:07.488457    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:15:12.488455 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:12.488415    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:15:14.645916 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.645883    2562 generic.go:356] "Generic (PLEG): container finished" podID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerID="5e63b621297f4d5fe18f2254bdd5d5b71e27f866998ea3fe564645a8feb231f1" exitCode=0
Jul 08 17:15:14.646340 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.645961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" event={"ID":"e352b067-8f1d-4604-b3ad-4731c9c07ef5","Type":"ContainerDied","Data":"5e63b621297f4d5fe18f2254bdd5d5b71e27f866998ea3fe564645a8feb231f1"}
Jul 08 17:15:14.687432 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.687411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:15:14.817622 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.817542    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls\") pod \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") "
Jul 08 17:15:14.817622 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.817613    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e352b067-8f1d-4604-b3ad-4731c9c07ef5-openshift-service-ca-bundle\") pod \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\" (UID: \"e352b067-8f1d-4604-b3ad-4731c9c07ef5\") "
Jul 08 17:15:14.817960 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.817939    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e352b067-8f1d-4604-b3ad-4731c9c07ef5-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e352b067-8f1d-4604-b3ad-4731c9c07ef5" (UID: "e352b067-8f1d-4604-b3ad-4731c9c07ef5"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:15:14.819551 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.819529    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e352b067-8f1d-4604-b3ad-4731c9c07ef5" (UID: "e352b067-8f1d-4604-b3ad-4731c9c07ef5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:15:14.918256 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.918220    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e352b067-8f1d-4604-b3ad-4731c9c07ef5-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:15:14.918256 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:14.918249    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e352b067-8f1d-4604-b3ad-4731c9c07ef5-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:15:15.650999 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:15.650969    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"
Jul 08 17:15:15.651475 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:15.650963    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc" event={"ID":"e352b067-8f1d-4604-b3ad-4731c9c07ef5","Type":"ContainerDied","Data":"337c4b92b28a55fe7a545ab67af7372196283c927925056f1b824f90b7b71c67"}
Jul 08 17:15:15.651475 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:15.651099    2562 scope.go:117] "RemoveContainer" containerID="5e63b621297f4d5fe18f2254bdd5d5b71e27f866998ea3fe564645a8feb231f1"
Jul 08 17:15:15.676922 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:15.672961    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"]
Jul 08 17:15:15.676922 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:15.676330    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-91fdc-d7f8d8f4c-q42gc"]
Jul 08 17:15:16.028439 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:15:16.028358    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" path="/var/lib/kubelet/pods/e352b067-8f1d-4604-b3ad-4731c9c07ef5/volumes"
Jul 08 17:18:03.445904 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:18:03.445768    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:18:03.452997 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:18:03.452972    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:22:18.739401 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:18.739366    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"]
Jul 08 17:22:18.739909 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:18.739574    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" containerID="cri-o://bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0" gracePeriod=30
Jul 08 17:22:18.912547 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:18.912512    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"]
Jul 08 17:22:18.912783 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:18.912739    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container" containerID="cri-o://f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda" gracePeriod=30
Jul 08 17:22:18.912911 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:18.912784    2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kube-rbac-proxy" containerID="cri-o://15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419" gracePeriod=30
Jul 08 17:22:19.389024 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:19.388990    2562 generic.go:356] "Generic (PLEG): container finished" podID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerID="15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419" exitCode=2
Jul 08 17:22:19.389197 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:19.389056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" event={"ID":"482ae0f4-77fe-41ca-b582-1cf28a9805f6","Type":"ContainerDied","Data":"15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419"}
Jul 08 17:22:21.363523 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.363481    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:22:21.756641 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.756618    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:22:21.832055 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.832027    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls\") pod \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") "
Jul 08 17:22:21.832220 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.832062    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gdpt\" (UniqueName: \"kubernetes.io/projected/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-api-access-5gdpt\") pod \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") "
Jul 08 17:22:21.832220 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.832087    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-rbac-proxy-sar-config\") pod \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\" (UID: \"482ae0f4-77fe-41ca-b582-1cf28a9805f6\") "
Jul 08 17:22:21.832466 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.832445    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "482ae0f4-77fe-41ca-b582-1cf28a9805f6" (UID: "482ae0f4-77fe-41ca-b582-1cf28a9805f6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:22:21.834207 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.834182    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "482ae0f4-77fe-41ca-b582-1cf28a9805f6" (UID: "482ae0f4-77fe-41ca-b582-1cf28a9805f6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:22:21.834207 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.834193    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-api-access-5gdpt" (OuterVolumeSpecName: "kube-api-access-5gdpt") pod "482ae0f4-77fe-41ca-b582-1cf28a9805f6" (UID: "482ae0f4-77fe-41ca-b582-1cf28a9805f6"). InnerVolumeSpecName "kube-api-access-5gdpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 17:22:21.933605 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.933525    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/482ae0f4-77fe-41ca-b582-1cf28a9805f6-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:22:21.933605 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.933554    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gdpt\" (UniqueName: \"kubernetes.io/projected/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-api-access-5gdpt\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:22:21.933605 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:21.933568    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/482ae0f4-77fe-41ca-b582-1cf28a9805f6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:22:22.402399 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.402364    2562 generic.go:356] "Generic (PLEG): container finished" podID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerID="f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda" exitCode=0
Jul 08 17:22:22.402858 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.402429    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"
Jul 08 17:22:22.402858 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.402442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" event={"ID":"482ae0f4-77fe-41ca-b582-1cf28a9805f6","Type":"ContainerDied","Data":"f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda"}
Jul 08 17:22:22.402858 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.402499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px" event={"ID":"482ae0f4-77fe-41ca-b582-1cf28a9805f6","Type":"ContainerDied","Data":"fc27166fa0b5af3bfb7be7d624837fe6c0345be6bf26b68c90fcb1cf3c06f4f8"}
Jul 08 17:22:22.402858 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.402524    2562 scope.go:117] "RemoveContainer" containerID="15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419"
Jul 08 17:22:22.412719 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.412699    2562 scope.go:117] "RemoveContainer" containerID="f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda"
Jul 08 17:22:22.419873 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.419853    2562 scope.go:117] "RemoveContainer" containerID="15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419"
Jul 08 17:22:22.420397 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:22:22.420376    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419\": container with ID starting with 15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419 not found: ID does not exist" containerID="15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419"
Jul 08 17:22:22.420485 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.420406    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419"} err="failed to get container status \"15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419\": rpc error: code = NotFound desc = could not find container \"15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419\": container with ID starting with 15279c47968b509fc0f15d87e88acb86a047dce663e9801840b737ba687b1419 not found: ID does not exist"
Jul 08 17:22:22.420485 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.420423    2562 scope.go:117] "RemoveContainer" containerID="f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda"
Jul 08 17:22:22.420735 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:22:22.420715    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda\": container with ID starting with f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda not found: ID does not exist" containerID="f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda"
Jul 08 17:22:22.420861 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.420745    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda"} err="failed to get container status \"f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda\": rpc error: code = NotFound desc = could not find container \"f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda\": container with ID starting with f3a012895616e299723f39e1f17ae60f19317c82f288dcdb57a7f5985586dfda not found: ID does not exist"
Jul 08 17:22:22.421366 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.421349    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"]
Jul 08 17:22:22.425634 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:22.425616    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73f94-predictor-68bc54bd54-h82px"]
Jul 08 17:22:24.028652 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:24.028619    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" path="/var/lib/kubelet/pods/482ae0f4-77fe-41ca-b582-1cf28a9805f6/volumes"
Jul 08 17:22:26.363942 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:26.363904    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:22:31.363902 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:31.363864    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:22:31.364296 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:31.363976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:22:35.140295 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:35.140267    2562 ???:1] "http: TLS handshake error from 10.0.128.57:44554: EOF"
Jul 08 17:22:35.143832 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:35.143798    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:35.993703 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:35.993671    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:36.363981 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:36.363947    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:22:36.809761 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:36.809683    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:37.587681 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:37.587652    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:38.381229 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:38.381200    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:39.186189 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:39.186165    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:40.007081 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:40.007051    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:40.794972 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:40.794947    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:41.363744 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:41.363707    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:22:41.578151 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:41.578121    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:42.389796 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:42.389772    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:43.170900 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:43.170857    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:43.957812 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:43.957779    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:44.780137 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:44.780111    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:45.628419 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:45.628382    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-73f94-65b6995f6-nlths_b4bee131-c4b3-45cf-a941-249bf8624a7a/switch-graph-73f94/0.log"
Jul 08 17:22:46.363089 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:46.363050    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 17:22:48.911389 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:48.911367    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:22:49.056665 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.056582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4bee131-c4b3-45cf-a941-249bf8624a7a-proxy-tls\") pod \"b4bee131-c4b3-45cf-a941-249bf8624a7a\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") "
Jul 08 17:22:49.056665 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.056655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4bee131-c4b3-45cf-a941-249bf8624a7a-openshift-service-ca-bundle\") pod \"b4bee131-c4b3-45cf-a941-249bf8624a7a\" (UID: \"b4bee131-c4b3-45cf-a941-249bf8624a7a\") "
Jul 08 17:22:49.057038 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.057017    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4bee131-c4b3-45cf-a941-249bf8624a7a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b4bee131-c4b3-45cf-a941-249bf8624a7a" (UID: "b4bee131-c4b3-45cf-a941-249bf8624a7a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 17:22:49.058578 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.058556    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4bee131-c4b3-45cf-a941-249bf8624a7a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b4bee131-c4b3-45cf-a941-249bf8624a7a" (UID: "b4bee131-c4b3-45cf-a941-249bf8624a7a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 17:22:49.158063 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.158014    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4bee131-c4b3-45cf-a941-249bf8624a7a-proxy-tls\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:22:49.158063 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.158062    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4bee131-c4b3-45cf-a941-249bf8624a7a-openshift-service-ca-bundle\") on node \"ip-10-0-132-182.ec2.internal\" DevicePath \"\""
Jul 08 17:22:49.511202 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.511163    2562 generic.go:356] "Generic (PLEG): container finished" podID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerID="bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0" exitCode=0
Jul 08 17:22:49.511477 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.511213    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"
Jul 08 17:22:49.511477 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.511218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" event={"ID":"b4bee131-c4b3-45cf-a941-249bf8624a7a","Type":"ContainerDied","Data":"bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0"}
Jul 08 17:22:49.511477 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.511269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths" event={"ID":"b4bee131-c4b3-45cf-a941-249bf8624a7a","Type":"ContainerDied","Data":"2a375144126c18ea0af508a90e0de429554256d7ff1e80ddf9097124622d8c2d"}
Jul 08 17:22:49.511477 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.511300    2562 scope.go:117] "RemoveContainer" containerID="bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0"
Jul 08 17:22:49.519589 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.519568    2562 scope.go:117] "RemoveContainer" containerID="bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0"
Jul 08 17:22:49.519860 ip-10-0-132-182 kubenswrapper[2562]: E0708 17:22:49.519836    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0\": container with ID starting with bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0 not found: ID does not exist" containerID="bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0"
Jul 08 17:22:49.519948 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.519865    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0"} err="failed to get container status \"bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0\": rpc error: code = NotFound desc = could not find container \"bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0\": container with ID starting with bb4492de98778ce621942719016f33228b2dfe2f6d86ea2ecd9eb05229b017d0 not found: ID does not exist"
Jul 08 17:22:49.530812 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.530790    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"]
Jul 08 17:22:49.534473 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:49.534455    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-73f94-65b6995f6-nlths"]
Jul 08 17:22:50.027645 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:50.027616    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" path="/var/lib/kubelet/pods/b4bee131-c4b3-45cf-a941-249bf8624a7a/volumes"
Jul 08 17:22:52.773124 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:52.773092    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mc44p_e967c6f7-5197-4648-8ce2-cffcd788026d/global-pull-secret-syncer/0.log"
Jul 08 17:22:52.845258 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:52.845229    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-89vm6_1c511300-5a57-43ae-b976-92821d4b9b59/konnectivity-agent/0.log"
Jul 08 17:22:52.925496 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:52.925452    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-182.ec2.internal_43f0a89a04d165574d4c36c955b6bc1e/haproxy/0.log"
Jul 08 17:22:56.855923 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:56.855882    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xgl2r_b45abeec-9848-44ca-b486-42f75a62b891/cluster-monitoring-operator/0.log"
Jul 08 17:22:57.046159 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:57.046118    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68h7g_cbca7f1b-2c6d-4985-848d-585fb86a554a/node-exporter/0.log"
Jul 08 17:22:57.072972 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:57.072949    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68h7g_cbca7f1b-2c6d-4985-848d-585fb86a554a/kube-rbac-proxy/0.log"
Jul 08 17:22:57.098299 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:57.098276    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68h7g_cbca7f1b-2c6d-4985-848d-585fb86a554a/init-textfile/0.log"
Jul 08 17:22:57.730056 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:57.730030    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-jf9ck_6ded0201-533f-4b2b-b270-9605778b186c/prometheus-operator-admission-webhook/0.log"
Jul 08 17:22:59.392311 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:59.392287    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-t88rq_b4342e11-7942-4b75-b0c5-d1b808b90bd2/networking-console-plugin/0.log"
Jul 08 17:22:59.972936 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:59.972906    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:22:59.977746 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:22:59.977721    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/3.log"
Jul 08 17:23:01.015985 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.015943    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-29gdq_5cc57719-0da2-430d-8f48-7ef9d9d774fd/volume-data-source-validator/0.log"
Jul 08 17:23:01.332387 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332360    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"]
Jul 08 17:23:01.332644 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332632    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kube-rbac-proxy"
Jul 08 17:23:01.332686 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332645    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kube-rbac-proxy"
Jul 08 17:23:01.332686 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332661    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kube-rbac-proxy"
Jul 08 17:23:01.332686 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332666    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kube-rbac-proxy"
Jul 08 17:23:01.332686 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332673    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container"
Jul 08 17:23:01.332686 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332678    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332689    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332695    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332703    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332708    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332717    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332722    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332770    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4bee131-c4b3-45cf-a941-249bf8624a7a" containerName="switch-graph-73f94"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332778    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kserve-container"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332785    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kube-rbac-proxy"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332791    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e352b067-8f1d-4604-b3ad-4731c9c07ef5" containerName="splitter-graph-91fdc"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332797    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="482ae0f4-77fe-41ca-b582-1cf28a9805f6" containerName="kube-rbac-proxy"
Jul 08 17:23:01.332851 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.332801    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb571fa-5d32-4a82-a0c2-34d9c659a80e" containerName="kserve-container"
Jul 08 17:23:01.346323 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.346295    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"]
Jul 08 17:23:01.346488 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.346398    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.348633 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.348608    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q5dxj\"/\"default-dockercfg-ttnqq\""
Jul 08 17:23:01.348803 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.348616    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q5dxj\"/\"kube-root-ca.crt\""
Jul 08 17:23:01.349744 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.349722    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q5dxj\"/\"openshift-service-ca.crt\""
Jul 08 17:23:01.449831 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.449800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-podres\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.450005 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.449849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-sys\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.450005 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.449942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-lib-modules\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.450005 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.449974    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb2dz\" (UniqueName: \"kubernetes.io/projected/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-kube-api-access-cb2dz\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.450005 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.449996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-proc\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550779 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-podres\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-sys\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-lib-modules\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb2dz\" (UniqueName: \"kubernetes.io/projected/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-kube-api-access-cb2dz\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-proc\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550936    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-sys\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550964    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-proc\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550937    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-podres\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.550971 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.550969    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-lib-modules\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.558705 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.558685    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb2dz\" (UniqueName: \"kubernetes.io/projected/a2a56793-2a9e-44c3-8a15-a7f5cb0c2422-kube-api-access-cb2dz\") pod \"perf-node-gather-daemonset-69pxq\" (UID: \"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422\") " pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.657873 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.657768    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:01.780920 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:01.780882    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"]
Jul 08 17:23:01.783579 ip-10-0-132-182 kubenswrapper[2562]: W0708 17:23:01.783553    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2a56793_2a9e_44c3_8a15_a7f5cb0c2422.slice/crio-9db92f0bc1636f788568efce1a1e15d8acdb2a436bb096b28ac487101eafff90 WatchSource:0}: Error finding container 9db92f0bc1636f788568efce1a1e15d8acdb2a436bb096b28ac487101eafff90: Status 404 returned error can't find the container with id 9db92f0bc1636f788568efce1a1e15d8acdb2a436bb096b28ac487101eafff90
Jul 08 17:23:02.036015 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.035939    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vctc7_539de09f-b3b6-4ed0-8e63-284856086968/dns/0.log"
Jul 08 17:23:02.057729 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.057703    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vctc7_539de09f-b3b6-4ed0-8e63-284856086968/kube-rbac-proxy/0.log"
Jul 08 17:23:02.189485 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.189454    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wzp4k_4f281f6c-3a96-491b-bb22-f8ae225f25b1/dns-node-resolver/0.log"
Jul 08 17:23:02.566768 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.566725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq" event={"ID":"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422","Type":"ContainerStarted","Data":"cdb66457ccb53ee56d76a95beef81775d8a1a230d3784c60b1d5eff8d2baa94c"}
Jul 08 17:23:02.566997 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.566799    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:02.566997 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.566835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq" event={"ID":"a2a56793-2a9e-44c3-8a15-a7f5cb0c2422","Type":"ContainerStarted","Data":"9db92f0bc1636f788568efce1a1e15d8acdb2a436bb096b28ac487101eafff90"}
Jul 08 17:23:02.582343 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.582297    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq" podStartSLOduration=1.58228216 podStartE2EDuration="1.58228216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:23:02.581288031 +0000 UTC m=+3003.143265413" watchObservedRunningTime="2026-07-08 17:23:02.58228216 +0000 UTC m=+3003.144259542"
Jul 08 17:23:02.693231 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:02.693209    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cpggg_ed690582-0a9e-4069-892c-78c07ea8066a/node-ca/0.log"
Jul 08 17:23:03.477068 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:03.476948    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:23:03.490845 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:03.482230    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tnz7d_84b7f28c-4a45-41ad-af20-44ca3821d67f/console-operator/2.log"
Jul 08 17:23:03.586100 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:03.586072    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7df5b6f649-fxv58_bda78aa0-0f42-40e1-bf1f-43d40bf5b0b3/router/0.log"
Jul 08 17:23:03.994090 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:03.994057    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-56gsq_7130273a-98c0-46d9-b77e-ed40a3bb73be/serve-healthcheck-canary/0.log"
Jul 08 17:23:04.399638 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:04.399609    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xh8sq_db950e2c-5b14-484d-941d-879cff4b3343/insights-operator/0.log"
Jul 08 17:23:04.400550 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:04.400532    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xh8sq_db950e2c-5b14-484d-941d-879cff4b3343/insights-operator/1.log"
Jul 08 17:23:04.487645 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:04.487615    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tcvsm_92d89032-2986-42a8-af24-0944878393fa/kube-rbac-proxy/0.log"
Jul 08 17:23:04.509296 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:04.509271    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tcvsm_92d89032-2986-42a8-af24-0944878393fa/exporter/0.log"
Jul 08 17:23:04.530758 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:04.530739    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tcvsm_92d89032-2986-42a8-af24-0944878393fa/extractor/0.log"
Jul 08 17:23:08.580325 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:08.580290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q5dxj/perf-node-gather-daemonset-69pxq"
Jul 08 17:23:10.946964 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:10.946942    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-nps8b_65d6ec16-0be4-47a2-9282-6961647a9ff1/migrator/0.log"
Jul 08 17:23:10.970374 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:10.970275    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-nps8b_65d6ec16-0be4-47a2-9282-6961647a9ff1/graceful-termination/0.log"
Jul 08 17:23:11.365876 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:11.365843    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-fqgl7_12d88f79-f370-45df-9935-944f43c1c1d7/kube-storage-version-migrator-operator/1.log"
Jul 08 17:23:11.366582 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:11.366568    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-fqgl7_12d88f79-f370-45df-9935-944f43c1c1d7/kube-storage-version-migrator-operator/0.log"
Jul 08 17:23:12.395300 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.395198    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6jjf5_1d57f4c7-60a7-47f1-8edb-3bcd4661c518/kube-multus/0.log"
Jul 08 17:23:12.591223 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.591194    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/kube-multus-additional-cni-plugins/0.log"
Jul 08 17:23:12.615628 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.615603    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/egress-router-binary-copy/0.log"
Jul 08 17:23:12.639093 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.639065    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/cni-plugins/0.log"
Jul 08 17:23:12.665200 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.665134    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/bond-cni-plugin/0.log"
Jul 08 17:23:12.690583 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.690567    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/routeoverride-cni/0.log"
Jul 08 17:23:12.715719 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.715694    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/whereabouts-cni-bincopy/0.log"
Jul 08 17:23:12.738359 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.738332    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sr2lk_6ca18ed4-8fcd-451e-bdf8-6e7d4f66aa4f/whereabouts-cni/0.log"
Jul 08 17:23:12.975462 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.975383    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-br9fz_cce8909d-08fb-45cb-a131-394cd09252a9/network-metrics-daemon/0.log"
Jul 08 17:23:12.994479 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:12.994459    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-br9fz_cce8909d-08fb-45cb-a131-394cd09252a9/kube-rbac-proxy/0.log"
Jul 08 17:23:14.055413 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.055387    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/ovn-controller/0.log"
Jul 08 17:23:14.073766 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.073745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/ovn-acl-logging/0.log"
Jul 08 17:23:14.097104 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.097079    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/kube-rbac-proxy-node/0.log"
Jul 08 17:23:14.120483 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.120460    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/kube-rbac-proxy-ovn-metrics/0.log"
Jul 08 17:23:14.142170 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.142138    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/northd/0.log"
Jul 08 17:23:14.167166 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.167146    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/nbdb/0.log"
Jul 08 17:23:14.195993 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.195971    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/sbdb/0.log"
Jul 08 17:23:14.247068 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:14.247045    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gx2vd_850ce6be-0856-49f4-9b25-cf2a0396d1a5/ovnkube-controller/0.log"
Jul 08 17:23:15.457316 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:15.457290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-989f8b8dc-9lf9x_cb132fbd-4b29-461b-b48b-7536187aee73/check-endpoints/0.log"
Jul 08 17:23:15.485804 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:15.485779    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-64tq5_f75e0ea9-22bb-495a-99bc-117ccfccde9b/network-check-target-container/0.log"
Jul 08 17:23:16.470401 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:16.470370    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-56rnv_114e39d2-544d-4713-9b42-efe1494a319f/iptables-alerter/0.log"
Jul 08 17:23:17.231052 ip-10-0-132-182 kubenswrapper[2562]: I0708 17:23:17.230985    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-7p92d_9cb737a7-e603-4199-86ec-e44ec1e67392/tuned/0.log"