Jul 27 17:27:20.238934 ip-10-0-132-235 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 17:27:20.682634 ip-10-0-132-235 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:27:20.682634 ip-10-0-132-235 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 17:27:20.682634 ip-10-0-132-235 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:27:20.682634 ip-10-0-132-235 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 17:27:20.682634 ip-10-0-132-235 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:27:20.684660 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.684570    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694139    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694158    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694162    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694166    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694169    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694172    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:27:20.694166 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694176    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694179    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694182    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694184    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694187    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694190    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694193    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694195    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694198    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694201    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694203    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694206    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694209    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694211    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694214    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694217    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694219    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694222    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694228    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:27:20.694416 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694232    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694236    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694239    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694242    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694244    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694247    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694249    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694252    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694255    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694257    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694260    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694263    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694265    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694268    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694270    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694274    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694277    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694279    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694282    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694285    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:27:20.694903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694288    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694290    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694293    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694295    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694299    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694302    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694305    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694307    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694310    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694313    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694315    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694318    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694320    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694324    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694328    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694330    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694333    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694336    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694339    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:27:20.695398 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694341    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694345    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694347    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694350    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694353    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694355    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694358    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694360    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694365    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694367    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694370    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694372    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694375    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694377    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694380    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694382    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694385    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694387    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694390    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694393    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:27:20.695874 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694396    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694398    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694401    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694403    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694406    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694410    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694412    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694416    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694419    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694422    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694425    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694428    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694431    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694433    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.694436    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695148    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695154    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695157    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695160    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:27:20.696390 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695162    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695167    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695171    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695173    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695176    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695178    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695181    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695184    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695186    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695189    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695191    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695194    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695197    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695200    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695202    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695205    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695207    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695210    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695214    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:27:20.696869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695217    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695221    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695224    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695226    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695230    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695232    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695234    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695237    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695239    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695242    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695245    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695249    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695253    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695255    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695259    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695261    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695264    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695266    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695269    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695273    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:27:20.697354 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695275    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695278    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695280    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695282    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695285    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695287    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695290    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695293    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695295    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695298    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695300    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695303    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695305    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695308    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695310    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695313    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695315    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695318    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695320    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695323    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:27:20.697867 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695325    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695328    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695330    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695332    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695337    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695341    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695345    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695348    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695351    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695353    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695356    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695358    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695361    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695364    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695366    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695369    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695371    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695374    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695376    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695379    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:27:20.698363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695382    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695385    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695387    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695390    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695393    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695395    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695398    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695400    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695403    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695405    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695408    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695410    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695413    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695415    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695418    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695420    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695804    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695810    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695814    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695817    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:27:20.698869 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695819    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695823    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695825    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695828    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695831    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695834    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695836    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695839    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695842    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695844    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695847    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695849    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695852    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695855    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695857    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695860    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695864    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695868    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695871    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:27:20.699387 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695874    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695877    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695881    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695884    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695887    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695890    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695893    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695896    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695899    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695902    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695906    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695909    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695911    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695914    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695917    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695919    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695922    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695925    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695927    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695930    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:27:20.699868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695932    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695934    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695937    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695940    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695942    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695945    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695948    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695950    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695953    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695955    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695958    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695960    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695963    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695965    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695968    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695971    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695974    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695976    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695979    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695981    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:27:20.700349 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695984    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695986    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695989    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695993    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695995    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.695998    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696000    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696003    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696005    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696008    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696012    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696015    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696018    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696020    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696023    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696025    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696028    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696031    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696034    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:27:20.700884 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696042    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696045    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696048    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696050    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696053    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696055    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696058    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696061    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696063    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696065    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696068    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696071    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696074    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696076    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696079    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696083    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696085    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696155    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696162    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696169    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 17:27:20.701363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696173    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696185    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696188    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696193    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696198    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696201    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696204    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696207    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696211    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696214    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696217    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696220    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696223    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696226    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696229    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696233    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696236    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696239    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696242    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696250    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696253    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696256    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696260    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 17:27:20.701890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696263    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696266    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696269    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696272    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696275    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696278    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696282    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696285    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696288    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696291    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696294    2563 flags.go:64] FLAG: --enable-server="true"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696297    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696301    2563 flags.go:64] FLAG: --event-burst="100"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696304    2563 flags.go:64] FLAG: --event-qps="50"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696307    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696310    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696313    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696316    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696319    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696325    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696328    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696330    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696333    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696337    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696339    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 17:27:20.702502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696342    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696345    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696348    2563 flags.go:64] FLAG: --feature-gates=""
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696352    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696355    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696357    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696362    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696365    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696368    2563 flags.go:64] FLAG: --help="false"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696371    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-132-235.ec2.internal"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696374    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696377    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696380    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696383    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696386    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696389    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696392    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696395    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696398    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696401    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696404    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696407    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696410    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696413    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 17:27:20.703159 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696416    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696419    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696423    2563 flags.go:64] FLAG: --lock-file=""
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696426    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696428    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696431    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696436    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696440    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696443    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696445    2563 flags.go:64] FLAG: --logging-format="text"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696448    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696451    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696454    2563 flags.go:64] FLAG: --manifest-url=""
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696457    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696462    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696465    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696469    2563 flags.go:64] FLAG: --max-pods="110"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696472    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696475    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696478    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696480    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696483    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696486    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696489    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696497    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 17:27:20.703786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696500    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696503    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696506    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696509    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696515    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696518    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696521    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696524    2563 flags.go:64] FLAG: --port="10250"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696527    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696532    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e991d14a1419e18a"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696535    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696538    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696541    2563 flags.go:64] FLAG: --register-node="true"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696543    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696547    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696551    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696554    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696556    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696560    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696563    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696566    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696568    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696572    2563 flags.go:64] FLAG: --runonce="false"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696575    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696578    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 17:27:20.704414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696581    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696584    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696587    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696590    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696592    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696595    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696598    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696601    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696603    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696606    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696609    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696612    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696615    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696620    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696623    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696625    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696633    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696635    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696638    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696641    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696644    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696647    2563 flags.go:64] FLAG: --v="2"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696651    2563 flags.go:64] FLAG: --version="false"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696656    2563 flags.go:64] FLAG: --vmodule=""
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696659    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 17:27:20.705046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.696663    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696788    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696792    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696796    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696800    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696803    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696806    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696808    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696811    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696814    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696816    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696819    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696822    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696825    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696829    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696832    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696835    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696838    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696841    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696843    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:27:20.705663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696846    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696849    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696851    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696855    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696858    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696860    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696863    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696866    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696869    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696871    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696874    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696877    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696880    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696883    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696885    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696888    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696892    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696895    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696914    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:27:20.706190 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696918    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696921    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696924    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696927    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696930    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696933    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696936    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696939    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696942    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696944    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696947    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696950    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696959    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696963    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696966    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696969    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696973    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696977    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696980    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:27:20.706649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696983    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696985    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696988    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696991    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696993    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.696996    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697001    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697004    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697006    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697009    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697012    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697015    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697018    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697020    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697023    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697026    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697028    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697031    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697034    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697036    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:27:20.707145 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697039    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697041    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697044    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697047    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697049    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697051    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697054    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697057    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697060    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697063    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697066    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697068    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697071    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697073    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697076    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697079    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697082    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697084    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697087    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697089    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:27:20.707628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697092    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.697094    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.698086    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.705139    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.705153    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.705167    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.705177    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705232    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705237    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705240    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705243    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705246    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705249    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705252    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705254    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:27:20.708209 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705257    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705260    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705262    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705265    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705268    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705272    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705276    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705279    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705281    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705284    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705287    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705290    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705292    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705295    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705298    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705300    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705303    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705305    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705308    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:27:20.708607 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705311    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705313    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705318    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705320    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705330    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705334    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705336    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705339    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705342    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705345    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705347    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705350    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705352    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705355    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705357    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705360    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705363    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705365    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705368    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705370    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:27:20.709085 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705373    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705376    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705378    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705381    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705383    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705386    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705388    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705390    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705393    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705396    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705398    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705401    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705404    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705406    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705409    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705413    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705416    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705418    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705421    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705423    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:27:20.709573 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705426    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705429    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705431    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705433    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705436    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705438    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705441    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705443    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705446    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705449    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705451    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705455    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705458    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705461    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705464    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705466    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705469    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705471    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705474    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:27:20.710102 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705476    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705481    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705484    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705487    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705490    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705494    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705496    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705499    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705502    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705505    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705508    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705511    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705514    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.705519    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:27:20.710569 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705628    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705632    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705635    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705639    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705641    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705644    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705648    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705652    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705655    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705658    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705661    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705663    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705666    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705668    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705686    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705691    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705694    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705697    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705699    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705702    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:27:20.710938 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705704    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705707    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705709    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705712    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705715    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705717    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705720    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705722    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705726    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705729    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705731    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705734    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705736    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705739    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705741    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705744    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705746    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705749    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705751    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705753    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:27:20.711410 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705756    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705758    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705761    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705763    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705765    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705768    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705770    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705773    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705776    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705778    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705781    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705784    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705786    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705789    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705791    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705793    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705796    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705799    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705801    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705804    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:27:20.711904 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705806    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705810    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705812    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705815    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705817    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705820    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705822    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705825    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705827    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705830    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705832    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705835    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705837    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705840    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705842    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705845    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705847    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705850    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705852    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705855    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:27:20.712381 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705858    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705860    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705863    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705865    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705868    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705870    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705873    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705875    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705878    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705880    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705883    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705886    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705889    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705891    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705895    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705898    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705902    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705905    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:27:20.712903 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:20.705908    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.705913    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.706015    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.708355    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.710118    2563 server.go:1019] "Starting client certificate rotation"
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.710223    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.710252    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:27:20.713359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.712856    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:27:20.714535 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.714508    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:27:20.726086 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.726069    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 27 17:27:20.729188 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.729175    2563 log.go:25] "Validated CRI v1 image API"
Jul 27 17:27:20.730002 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.729978    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 17:27:20.732576 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.732556    2563 fs.go:135] Filesystem UUIDs: map[171a762a-73f9-4936-9a50-a71d608acb50:/dev/nvme0n1p3 34b034fd-d25b-4fa3-a4e3-36966959d5b7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 27 17:27:20.732669 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.732576    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 17:27:20.738423 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.738309    2563 manager.go:216] Machine: {Timestamp:2026-07-27 17:27:20.736881303 +0000 UTC m=+0.362276711 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099607 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2348ae2965539980aa0629b2bb2472 SystemUUID:ec2348ae-2965-5399-80aa-0629b2bb2472 BootID:c8777663-06b3-473e-b17d-88fbb0b37363 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:08:d9:f2:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:08:d9:f2:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:13:6f:dd:f6:02 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 17:27:20.738423 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.738412    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 17:27:20.738580 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.738506    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 17:27:20.742087 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742064    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 17:27:20.742238 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742224    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:27:20.742311 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742088    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 17:27:20.742964 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742952    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 17:27:20.743008 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742967    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 17:27:20.743008 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742974    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:27:20.743008 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.742988    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:27:20.744908 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.744897    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 17:27:20.749514 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749502    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:27:20.749623 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749614    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 17:27:20.749653 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749643    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 17:27:20.749653 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749652    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 17:27:20.749756 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749666    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 17:27:20.749756 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749688    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 27 17:27:20.749756 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.749703    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 17:27:20.750051 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.750038    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:27:20.750082 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.750063    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:27:20.752329 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.752314    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 17:27:20.754037 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754021    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 17:27:20.754104 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754050    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 17:27:20.754897 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754886    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 17:27:20.754931 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754902    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 17:27:20.754931 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754908    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 17:27:20.754931 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754914    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 17:27:20.754931 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754922    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 17:27:20.754931 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754927    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 17:27:20.755068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754935    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 17:27:20.755068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754940    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 17:27:20.755068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754946    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 17:27:20.755068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754951    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 17:27:20.755068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754959    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 17:27:20.755068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.754965    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 17:27:20.755804 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.755794    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 17:27:20.755871 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.755807    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 17:27:20.758795 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.758777    2563 server.go:1268] "Started kubelet"
Jul 27 17:27:20.758959 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.758927    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 17:27:20.763837 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.758868    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 17:27:20.763837 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.762653    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 17:27:20.763155 ip-10-0-132-235 systemd[1]: Started Kubernetes Kubelet.
Jul 27 17:27:20.765072 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.765046    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 17:27:20.766518 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.766495    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 17:27:20.767153 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.767137    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 17:27:20.767507 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.767483    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 17:27:20.767668 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.767657    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 17:27:20.767809 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.767745    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 17:27:20.768350 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.767226    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-235.ec2.internal.18c6359aadf54dda  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-235.ec2.internal,UID:ip-10-0-132-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-235.ec2.internal,},FirstTimestamp:2026-07-27 17:27:20.758750682 +0000 UTC m=+0.384146090,LastTimestamp:2026-07-27 17:27:20.758750682 +0000 UTC m=+0.384146090,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-235.ec2.internal,}"
Jul 27 17:27:20.776104 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.776084    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 17:27:20.776223 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.776089    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 17:27:20.776724 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.776705    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:20.777192 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777175    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 17:27:20.777192 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777192    2563 factory.go:55] Registering systemd factory
Jul 27 17:27:20.777330 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777200    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 27 17:27:20.777330 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777235    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 17:27:20.777330 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777247    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 17:27:20.777330 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777264    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 17:27:20.777516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777365    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 17:27:20.777516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777378    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 17:27:20.777516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777419    2563 factory.go:153] Registering CRI-O factory
Jul 27 17:27:20.777516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777432    2563 factory.go:223] Registration of the crio container factory successfully
Jul 27 17:27:20.777516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777454    2563 factory.go:103] Registering Raw factory
Jul 27 17:27:20.777516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777466    2563 manager.go:1196] Started watching for new ooms in manager
Jul 27 17:27:20.777871 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.777858    2563 manager.go:349] Starting recovery of all containers
Jul 27 17:27:20.778943 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.778921    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 17:27:20.779032 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.778950    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 17:27:20.790481 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.790461    2563 manager.go:354] Recovery completed
Jul 27 17:27:20.790635 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.790601    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tbgxg"
Jul 27 17:27:20.793007 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.791935    2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 27 17:27:20.794761 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.794749    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.796401 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796385    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.796456 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796415    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.796456 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796429    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:20.796942 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796925    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 17:27:20.796942 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796942    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 17:27:20.797062 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796961    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:27:20.797062 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.796942    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.797856 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.797841    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.797928 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.797865    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.797928 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.797875    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:20.799189 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.799173    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tbgxg"
Jul 27 17:27:20.799262 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.799220    2563 policy_none.go:49] "None policy: Start"
Jul 27 17:27:20.799262 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.799239    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 17:27:20.799262 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.799253    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 17:27:20.802535 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.802391    2563 policy_none.go:47] "Start"
Jul 27 17:27:20.805732 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.805640    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-235.ec2.internal.18c6359ab033c9c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-235.ec2.internal,UID:ip-10-0-132-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-235.ec2.internal,},FirstTimestamp:2026-07-27 17:27:20.796400072 +0000 UTC m=+0.421795480,LastTimestamp:2026-07-27 17:27:20.796400072 +0000 UTC m=+0.421795480,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-235.ec2.internal,}"
Jul 27 17:27:20.828736 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.828721    2563 manager.go:337] "Starting Device Plugin manager"
Jul 27 17:27:20.828815 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.828759    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 17:27:20.828815 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.828773    2563 server.go:87] "Starting device plugin registration server"
Jul 27 17:27:20.829015 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.829003    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 17:27:20.829086 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.829020    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 17:27:20.829133 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.829099    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 17:27:20.829188 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.829171    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 17:27:20.829188 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.829179    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 17:27:20.829790 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.829766    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 17:27:20.829907 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.829816    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:20.875195 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.875162    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 17:27:20.877315 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.876333    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 17:27:20.877315 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.876372    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 17:27:20.877315 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.876398    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 17:27:20.877315 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.876437    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 17:27:20.878929 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.878913    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:27:20.930005 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.929982    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.930846 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.930831    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.930914 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.930857    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.930914 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.930868    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:20.930914 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.930891    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-235.ec2.internal"
Jul 27 17:27:20.943012 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.942970    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-235.ec2.internal"
Jul 27 17:27:20.943012 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.942990    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-235.ec2.internal\": node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:20.965614 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:20.965594    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:20.976653 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.976635    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"]
Jul 27 17:27:20.976727 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.976713    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.979527 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.979514    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.979581 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.979539    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.979581 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.979552    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:20.981831 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.981819    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.981936 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.981922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:20.981969 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.981952    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.982439 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.982426    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.982494 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.982441    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.982494 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.982451    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.982494 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.982461    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:20.982494 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.982465    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.982494 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.982480    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:20.984767 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.984748    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:20.984810 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.984778    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:20.985385 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.985369    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:20.985439 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.985400    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:20.985439 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:20.985415    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:21.007287 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.007269    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-235.ec2.internal\" not found" node="ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.011544 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.011530    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-235.ec2.internal\" not found" node="ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.066199 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.066171    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:21.078997 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.078978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/421b99f667c7b46ca0f75b4af30961f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal\" (UID: \"421b99f667c7b46ca0f75b4af30961f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.079061 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.079002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421b99f667c7b46ca0f75b4af30961f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal\" (UID: \"421b99f667c7b46ca0f75b4af30961f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.079061 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.079020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1eab52b5abcc336338b0570279cda8a5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-235.ec2.internal\" (UID: \"1eab52b5abcc336338b0570279cda8a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.167127 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.167096    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:21.179452 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.179429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/421b99f667c7b46ca0f75b4af30961f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal\" (UID: \"421b99f667c7b46ca0f75b4af30961f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.179523 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.179457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421b99f667c7b46ca0f75b4af30961f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal\" (UID: \"421b99f667c7b46ca0f75b4af30961f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.179523 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.179478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1eab52b5abcc336338b0570279cda8a5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-235.ec2.internal\" (UID: \"1eab52b5abcc336338b0570279cda8a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.179523 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.179505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1eab52b5abcc336338b0570279cda8a5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-235.ec2.internal\" (UID: \"1eab52b5abcc336338b0570279cda8a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.179639 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.179532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/421b99f667c7b46ca0f75b4af30961f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal\" (UID: \"421b99f667c7b46ca0f75b4af30961f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.179639 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.179539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421b99f667c7b46ca0f75b4af30961f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal\" (UID: \"421b99f667c7b46ca0f75b4af30961f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.267857 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.267818    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:21.309285 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.309264    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.314888 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:21.314602    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:21.368843 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.368818    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:21.469385 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.469362    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:21.569912 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:21.569851    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:23.569758 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.569738    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:27:23.585134 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:23.585114    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:23.623648 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.623632    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 17:27:23.624107 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.623739    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:27:23.624107 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.623774    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:27:23.685801 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:23.685779    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:23.720464 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.720432    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 17:22:22 +0000 UTC" deadline="2028-04-28 15:07:13.002095936 +0000 UTC"
Jul 27 17:27:23.720464 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.720465    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15381h39m49.281635127s"
Jul 27 17:27:23.785924 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:23.785881    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:23.798663 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:23.798633    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421b99f667c7b46ca0f75b4af30961f3.slice/crio-24049339ce489b21a192a7b86516e23cc8ad29e95b4989153754af2f1029160a WatchSource:0}: Error finding container 24049339ce489b21a192a7b86516e23cc8ad29e95b4989153754af2f1029160a: Status 404 returned error can't find the container with id 24049339ce489b21a192a7b86516e23cc8ad29e95b4989153754af2f1029160a
Jul 27 17:27:23.799255 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:23.799234    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eab52b5abcc336338b0570279cda8a5.slice/crio-261ba018befe0a2cb7c0ded3263b872b4022a4bd05d146bdbae00963c9612da4 WatchSource:0}: Error finding container 261ba018befe0a2cb7c0ded3263b872b4022a4bd05d146bdbae00963c9612da4: Status 404 returned error can't find the container with id 261ba018befe0a2cb7c0ded3263b872b4022a4bd05d146bdbae00963c9612da4
Jul 27 17:27:23.805605 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.805591    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:27:23.824118 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.824099    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:27:23.880614 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.880593    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:27:23.881662 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.881646    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:27:23.881731 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.881673    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:27:23.881731 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:23.881684    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:27:23.886174 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:23.886160    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:23.986403 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:23.986380    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:24.086938 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.086899    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-235.ec2.internal\" not found"
Jul 27 17:27:24.170459 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.169296    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:27:24.192104 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.192084    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:24.206923 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.206905    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:27:24.208018 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.208005    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal"
Jul 27 17:27:24.214579 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.214566    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:27:24.665419 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.665391    2563 apiserver.go:52] "Watching apiserver"
Jul 27 17:27:24.671097 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.671073    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 17:27:24.671454 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.671433    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-rjcvx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal","openshift-multus/multus-additional-cni-plugins-hzgdr","openshift-multus/multus-r82m9","openshift-network-operator/iptables-alerter-grtsm","kube-system/global-pull-secret-syncer-phrw4","kube-system/konnectivity-agent-jxl2n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx","openshift-image-registry/node-ca-wk5wk","openshift-multus/network-metrics-daemon-n44f9","openshift-network-diagnostics/network-check-target-dxf4t","openshift-ovn-kubernetes/ovnkube-node-cxgdr","kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal","openshift-cluster-node-tuning-operator/tuned-b4wnd"]
Jul 27 17:27:24.674127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.674108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.675935 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.675908    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 17:27:24.676058 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.676042    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.676120 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.676109    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.676183 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.676164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v5lbk\""
Jul 27 17:27:24.676582 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.676563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.678068 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.678044    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nqrvj\""
Jul 27 17:27:24.678149 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.678107    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.678401 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.678379    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.678983 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.678962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.680638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.680619    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.680728 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.680643    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g9vfp\""
Jul 27 17:27:24.680921 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.680906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.681185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.680913    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 17:27:24.681185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.681035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 17:27:24.681639 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.681267    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 17:27:24.681639 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.681303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.682828 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.682804    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 17:27:24.682937 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.682915    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n4wh7\""
Jul 27 17:27:24.683431 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.683412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:24.683526 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.683479    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:24.686435 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.686416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.688510 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.688491    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 17:27:24.688600 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.688546    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 17:27:24.688600 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.688575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-259qw\""
Jul 27 17:27:24.688822 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.688804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.690398 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.690379    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.690486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.690473    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-27s85\""
Jul 27 17:27:24.690542 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.690487    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.691487 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.691468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.691753 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.691734    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 17:27:24.693275 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.693095    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.693275 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.693147    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 17:27:24.693433 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.693384    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cz8hx\""
Jul 27 17:27:24.693489 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.693459    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.693757 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.693740    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.693846 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.693797    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:24.696414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.696377    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:24.696511 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.696461    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:24.699062 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.699045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.701450 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.701430    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.701690 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.701477    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 17:27:24.701762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.701539    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mqmb8\""
Jul 27 17:27:24.701827 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.701547    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.701982 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.701965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.702518 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.702502    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:27:24.704014 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.703991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w7mcf\""
Jul 27 17:27:24.704103 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704055    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 17:27:24.704103 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 17:27:24.704215 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh54z\" (UniqueName: \"kubernetes.io/projected/4d69785c-df22-4d63-86f2-e60bfb5297f1-kube-api-access-gh54z\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.704275 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-host\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.704275 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-socket-dir-parent\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.704275 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-kubelet\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.704275 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-sys\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.704498 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-serviceca\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.704498 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04a2888d-ead2-4657-a191-b7fe4d26c689-kubelet-config\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.704498 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e-agent-certs\") pod \"konnectivity-agent-jxl2n\" (UID: \"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e\") " pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-k8s-cni-cncf-io\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-kubernetes\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-var-lib-kubelet\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-tuned\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqs5\" (UniqueName: \"kubernetes.io/projected/1d4ac820-1268-4587-b1a8-fa9ad85ba912-kube-api-access-vgqs5\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-os-release\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-socket-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704733    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 17:27:24.704762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-run\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-cni-bin\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e-konnectivity-ca\") pod \"konnectivity-agent-jxl2n\" (UID: \"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e\") " pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-system-cni-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-conf-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbjr9\" (UniqueName: \"kubernetes.io/projected/c74e1051-81e6-4b86-b04e-11df18125251-kube-api-access-kbjr9\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-cnibin\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6xb8\" (UniqueName: \"kubernetes.io/projected/78a9220d-c136-489e-84db-36da1c4423dc-kube-api-access-m6xb8\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.704850    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705006    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-etc-selinux\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-sys-fs\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-cni-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.705181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-netns\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysctl-d\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-os-release\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d4ac820-1268-4587-b1a8-fa9ad85ba912-tmp\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-system-cni-dir\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a9220d-c136-489e-84db-36da1c4423dc-hosts-file\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a9220d-c136-489e-84db-36da1c4423dc-tmp-dir\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-device-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysconfig\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-systemd\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-multus-certs\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.705833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6nd8\" (UniqueName: \"kubernetes.io/projected/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-kube-api-access-r6nd8\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-modprobe-d\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/141a7e0f-fdc5-4152-9626-c6387bece93a-cni-binary-copy\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttqqd\" (UniqueName: \"kubernetes.io/projected/141a7e0f-fdc5-4152-9626-c6387bece93a-kube-api-access-ttqqd\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-host\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-daemon-config\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-registration-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysctl-conf\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-hostroot\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-lib-modules\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-cnibin\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.705984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-cni-multus\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.706011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.706045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-etc-kubernetes\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.706700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.706122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04a2888d-ead2-4657-a191-b7fe4d26c689-dbus\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.728926 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.728909    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5257t"
Jul 27 17:27:24.739769 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.739748    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5257t"
Jul 27 17:27:24.792964 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.792939    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 17:27:24.795170 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.795126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal" event={"ID":"1eab52b5abcc336338b0570279cda8a5","Type":"ContainerStarted","Data":"261ba018befe0a2cb7c0ded3263b872b4022a4bd05d146bdbae00963c9612da4"}
Jul 27 17:27:24.796185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.796160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal" event={"ID":"421b99f667c7b46ca0f75b4af30961f3","Type":"ContainerStarted","Data":"24049339ce489b21a192a7b86516e23cc8ad29e95b4989153754af2f1029160a"}
Jul 27 17:27:24.806366 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-socket-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.806475 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-run\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.806475 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-cni-bin\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.806475 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr8zp\" (UniqueName: \"kubernetes.io/projected/269531c4-3fdf-4ceb-8046-2228ac3c143d-kube-api-access-tr8zp\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:24.806475 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e-konnectivity-ca\") pod \"konnectivity-agent-jxl2n\" (UID: \"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e\") " pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.806475 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-cni-bin\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-run\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-system-cni-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-socket-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-conf-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-system-cni-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-conf-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbjr9\" (UniqueName: \"kubernetes.io/projected/c74e1051-81e6-4b86-b04e-11df18125251-kube-api-access-kbjr9\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-cnibin\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6xb8\" (UniqueName: \"kubernetes.io/projected/78a9220d-c136-489e-84db-36da1c4423dc-kube-api-access-m6xb8\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-node-log\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-cnibin\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-run-ovn-kubernetes\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:24.806723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-etc-selinux\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-sys-fs\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-cni-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-netns\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysctl-d\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-sys-fs\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-kubelet\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-etc-selinux\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-netns\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-ovnkube-script-lib\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-os-release\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysctl-d\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807019    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-os-release\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.807355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d4ac820-1268-4587-b1a8-fa9ad85ba912-tmp\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e-konnectivity-ca\") pod \"konnectivity-agent-jxl2n\" (UID: \"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e\") " pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-system-cni-dir\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-system-cni-dir\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.806968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-cni-dir\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a9220d-c136-489e-84db-36da1c4423dc-hosts-file\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a9220d-c136-489e-84db-36da1c4423dc-tmp-dir\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807320    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-device-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysconfig\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a9220d-c136-489e-84db-36da1c4423dc-hosts-file\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-systemd\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-systemd\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807499    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-device-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.808162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysconfig\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-multus-certs\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-cni-netd\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-multus-certs\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a9220d-c136-489e-84db-36da1c4423dc-tmp-dir\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6nd8\" (UniqueName: \"kubernetes.io/projected/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-kube-api-access-r6nd8\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.807862    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-modprobe-d\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/141a7e0f-fdc5-4152-9626-c6387bece93a-cni-binary-copy\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.807927    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:25.307907475 +0000 UTC m=+3.018643137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttqqd\" (UniqueName: \"kubernetes.io/projected/141a7e0f-fdc5-4152-9626-c6387bece93a-kube-api-access-ttqqd\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.807973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-cni-bin\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9066e682-b257-499f-8c53-b61cc205317b-ovn-node-metrics-cert\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-host\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.808998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-daemon-config\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d69785c-df22-4d63-86f2-e60bfb5297f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc7dp\" (UniqueName: \"kubernetes.io/projected/9066e682-b257-499f-8c53-b61cc205317b-kube-api-access-zc7dp\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-registration-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysctl-conf\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-hostroot\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-run-netns\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-ovn\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-lib-modules\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c74e1051-81e6-4b86-b04e-11df18125251-registration-dir\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-cnibin\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-cni-multus\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808494    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-modprobe-d\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/141a7e0f-fdc5-4152-9626-c6387bece93a-cni-binary-copy\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-log-socket\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-cnibin\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.809802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-cni-multus\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-etc-kubernetes\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-slash\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-ovnkube-config\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-env-overrides\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-hostroot\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04a2888d-ead2-4657-a191-b7fe4d26c689-dbus\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh54z\" (UniqueName: \"kubernetes.io/projected/4d69785c-df22-4d63-86f2-e60bfb5297f1-kube-api-access-gh54z\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-host\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-socket-dir-parent\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04a2888d-ead2-4657-a191-b7fe4d26c689-dbus\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-host\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-daemon-config\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-kubelet\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-systemd-units\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-var-lib-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.810501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808940    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-lib-modules\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-sys\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-sysctl-conf\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.808982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-etc-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-multus-socket-dir-parent\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-var-lib-kubelet\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8193031b-4bdb-4756-9607-4fe816a48535-iptables-alerter-script\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-sys\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-host\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jnjd\" (UniqueName: \"kubernetes.io/projected/8193031b-4bdb-4756-9607-4fe816a48535-kube-api-access-7jnjd\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-etc-kubernetes\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-serviceca\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04a2888d-ead2-4657-a191-b7fe4d26c689-kubelet-config\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04a2888d-ead2-4657-a191-b7fe4d26c689-kubelet-config\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e-agent-certs\") pod \"konnectivity-agent-jxl2n\" (UID: \"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e\") " pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-k8s-cni-cncf-io\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-systemd\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.811268 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8193031b-4bdb-4756-9607-4fe816a48535-host-slash\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-kubernetes\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-var-lib-kubelet\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d69785c-df22-4d63-86f2-e60bfb5297f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-tuned\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqs5\" (UniqueName: \"kubernetes.io/projected/1d4ac820-1268-4587-b1a8-fa9ad85ba912-kube-api-access-vgqs5\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809493    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-kubernetes\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-os-release\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d4ac820-1268-4587-b1a8-fa9ad85ba912-var-lib-kubelet\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-serviceca\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-os-release\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.809624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/141a7e0f-fdc5-4152-9626-c6387bece93a-host-run-k8s-cni-cncf-io\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.810988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d4ac820-1268-4587-b1a8-fa9ad85ba912-tmp\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.811814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d4ac820-1268-4587-b1a8-fa9ad85ba912-etc-tuned\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.812193 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.812087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e-agent-certs\") pod \"konnectivity-agent-jxl2n\" (UID: \"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e\") " pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:24.813873 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.813820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbjr9\" (UniqueName: \"kubernetes.io/projected/c74e1051-81e6-4b86-b04e-11df18125251-kube-api-access-kbjr9\") pod \"aws-ebs-csi-driver-node-cdtjx\" (UID: \"c74e1051-81e6-4b86-b04e-11df18125251\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:24.816344 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.816310    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:24.816445 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.816360    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:24.816445 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.816376    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:24.816445 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.816435    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:27:25.316419156 +0000 UTC m=+3.027154805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:24.818896 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.818860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6nd8\" (UniqueName: \"kubernetes.io/projected/56a1b52a-257c-48a7-8a73-e59e27e1a1d3-kube-api-access-r6nd8\") pod \"node-ca-wk5wk\" (UID: \"56a1b52a-257c-48a7-8a73-e59e27e1a1d3\") " pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.819258 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.819235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqs5\" (UniqueName: \"kubernetes.io/projected/1d4ac820-1268-4587-b1a8-fa9ad85ba912-kube-api-access-vgqs5\") pod \"tuned-b4wnd\" (UID: \"1d4ac820-1268-4587-b1a8-fa9ad85ba912\") " pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:24.819371 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.819238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttqqd\" (UniqueName: \"kubernetes.io/projected/141a7e0f-fdc5-4152-9626-c6387bece93a-kube-api-access-ttqqd\") pod \"multus-r82m9\" (UID: \"141a7e0f-fdc5-4152-9626-c6387bece93a\") " pod="openshift-multus/multus-r82m9"
Jul 27 17:27:24.819627 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.819610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6xb8\" (UniqueName: \"kubernetes.io/projected/78a9220d-c136-489e-84db-36da1c4423dc-kube-api-access-m6xb8\") pod \"node-resolver-rjcvx\" (UID: \"78a9220d-c136-489e-84db-36da1c4423dc\") " pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:24.821131 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.821113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh54z\" (UniqueName: \"kubernetes.io/projected/4d69785c-df22-4d63-86f2-e60bfb5297f1-kube-api-access-gh54z\") pod \"multus-additional-cni-plugins-hzgdr\" (UID: \"4d69785c-df22-4d63-86f2-e60bfb5297f1\") " pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:24.832464 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.832443    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:27:24.910380 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-cni-bin\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910500 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9066e682-b257-499f-8c53-b61cc205317b-ovn-node-metrics-cert\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910500 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc7dp\" (UniqueName: \"kubernetes.io/projected/9066e682-b257-499f-8c53-b61cc205317b-kube-api-access-zc7dp\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910500 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-run-netns\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910500 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-ovn\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910500 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-log-socket\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-cni-bin\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-slash\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-run-netns\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-ovnkube-config\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-slash\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-ovn\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-env-overrides\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-log-socket\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-systemd-units\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-var-lib-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-etc-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-systemd-units\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8193031b-4bdb-4756-9607-4fe816a48535-iptables-alerter-script\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jnjd\" (UniqueName: \"kubernetes.io/projected/8193031b-4bdb-4756-9607-4fe816a48535-kube-api-access-7jnjd\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-var-lib-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-systemd\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.910768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-etc-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-systemd\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8193031b-4bdb-4756-9607-4fe816a48535-host-slash\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr8zp\" (UniqueName: \"kubernetes.io/projected/269531c4-3fdf-4ceb-8046-2228ac3c143d-kube-api-access-tr8zp\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8193031b-4bdb-4756-9607-4fe816a48535-host-slash\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-node-log\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.910995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-run-ovn-kubernetes\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-node-log\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-run-ovn-kubernetes\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-kubelet\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-env-overrides\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-ovnkube-script-lib\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-ovnkube-config\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-run-openvswitch\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-kubelet\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.911152    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:24.911624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.912253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-cni-netd\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.912253 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:24.911239    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:27:25.411224213 +0000 UTC m=+3.121959871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:24.912253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-cni-netd\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.912253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8193031b-4bdb-4756-9607-4fe816a48535-iptables-alerter-script\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.912253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9066e682-b257-499f-8c53-b61cc205317b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.912253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.911567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9066e682-b257-499f-8c53-b61cc205317b-ovnkube-script-lib\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.912876 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.912848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9066e682-b257-499f-8c53-b61cc205317b-ovn-node-metrics-cert\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.919050 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.918998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr8zp\" (UniqueName: \"kubernetes.io/projected/269531c4-3fdf-4ceb-8046-2228ac3c143d-kube-api-access-tr8zp\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:24.919325 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.919302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jnjd\" (UniqueName: \"kubernetes.io/projected/8193031b-4bdb-4756-9607-4fe816a48535-kube-api-access-7jnjd\") pod \"iptables-alerter-grtsm\" (UID: \"8193031b-4bdb-4756-9607-4fe816a48535\") " pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:24.919427 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.919309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc7dp\" (UniqueName: \"kubernetes.io/projected/9066e682-b257-499f-8c53-b61cc205317b-kube-api-access-zc7dp\") pod \"ovnkube-node-cxgdr\" (UID: \"9066e682-b257-499f-8c53-b61cc205317b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:24.985168 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.985144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wk5wk"
Jul 27 17:27:24.991765 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:24.991750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b4wnd"
Jul 27 17:27:25.003964 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.003941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hzgdr"
Jul 27 17:27:25.007720 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.007698    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r82m9"
Jul 27 17:27:25.009619 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.009598    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4ac820_1268_4587_b1a8_fa9ad85ba912.slice/crio-aef98d3bf88d0c066103d460b87a90232ee914a38cf5c71e652d545ff771721b WatchSource:0}: Error finding container aef98d3bf88d0c066103d460b87a90232ee914a38cf5c71e652d545ff771721b: Status 404 returned error can't find the container with id aef98d3bf88d0c066103d460b87a90232ee914a38cf5c71e652d545ff771721b
Jul 27 17:27:25.011715 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.011693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a1b52a_257c_48a7_8a73_e59e27e1a1d3.slice/crio-caf8a839d7829dfce585b498eb78246e55ea7590bc85ac0b938fa484d4ea3684 WatchSource:0}: Error finding container caf8a839d7829dfce585b498eb78246e55ea7590bc85ac0b938fa484d4ea3684: Status 404 returned error can't find the container with id caf8a839d7829dfce585b498eb78246e55ea7590bc85ac0b938fa484d4ea3684
Jul 27 17:27:25.014208 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.014187    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:25.015181 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.015158    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d69785c_df22_4d63_86f2_e60bfb5297f1.slice/crio-fcb6826b002531e743c3b4e43f22bc7ab7f453616c43002c3b74d3760d5ee67f WatchSource:0}: Error finding container fcb6826b002531e743c3b4e43f22bc7ab7f453616c43002c3b74d3760d5ee67f: Status 404 returned error can't find the container with id fcb6826b002531e743c3b4e43f22bc7ab7f453616c43002c3b74d3760d5ee67f
Jul 27 17:27:25.017136 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.017117    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141a7e0f_fdc5_4152_9626_c6387bece93a.slice/crio-ef5a3e094406ae075babece627bea08360e1fb264535f6ff6a2baefc30d7b2fd WatchSource:0}: Error finding container ef5a3e094406ae075babece627bea08360e1fb264535f6ff6a2baefc30d7b2fd: Status 404 returned error can't find the container with id ef5a3e094406ae075babece627bea08360e1fb264535f6ff6a2baefc30d7b2fd
Jul 27 17:27:25.020267 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.020227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjcvx"
Jul 27 17:27:25.025204 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.025180    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19ebf9f_ccb5_400e_b6f0_d2f8a2fdb44e.slice/crio-bfee2dd2406a2d8ef26dfc99bfa15c93817bc8b39a123a75135ee883c2902547 WatchSource:0}: Error finding container bfee2dd2406a2d8ef26dfc99bfa15c93817bc8b39a123a75135ee883c2902547: Status 404 returned error can't find the container with id bfee2dd2406a2d8ef26dfc99bfa15c93817bc8b39a123a75135ee883c2902547
Jul 27 17:27:25.027079 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.027062    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx"
Jul 27 17:27:25.030411 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.030385    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a9220d_c136_489e_84db_36da1c4423dc.slice/crio-96c16345e8b3e43dbc2dbdaaa87179a1d29bd5b10210fd34d4e04bd253fe570d WatchSource:0}: Error finding container 96c16345e8b3e43dbc2dbdaaa87179a1d29bd5b10210fd34d4e04bd253fe570d: Status 404 returned error can't find the container with id 96c16345e8b3e43dbc2dbdaaa87179a1d29bd5b10210fd34d4e04bd253fe570d
Jul 27 17:27:25.034943 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.034926    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-grtsm"
Jul 27 17:27:25.036649 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.036622    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74e1051_81e6_4b86_b04e_11df18125251.slice/crio-17a849b8b7ea08d843ef34e763d8d1e94a1b03a1e21e7d048d64d38b2b84cf91 WatchSource:0}: Error finding container 17a849b8b7ea08d843ef34e763d8d1e94a1b03a1e21e7d048d64d38b2b84cf91: Status 404 returned error can't find the container with id 17a849b8b7ea08d843ef34e763d8d1e94a1b03a1e21e7d048d64d38b2b84cf91
Jul 27 17:27:25.039893 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.039878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:25.043714 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.043691    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8193031b_4bdb_4756_9607_4fe816a48535.slice/crio-36cfd7499c7b3de675fb91b888c9351e07564a075a62015ea373db33176577a3 WatchSource:0}: Error finding container 36cfd7499c7b3de675fb91b888c9351e07564a075a62015ea373db33176577a3: Status 404 returned error can't find the container with id 36cfd7499c7b3de675fb91b888c9351e07564a075a62015ea373db33176577a3
Jul 27 17:27:25.049894 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:25.049868    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9066e682_b257_499f_8c53_b61cc205317b.slice/crio-9539426ab59cbfc855a24fe93a71ddefdd0380d861881e7ac97d10340cc5ec1d WatchSource:0}: Error finding container 9539426ab59cbfc855a24fe93a71ddefdd0380d861881e7ac97d10340cc5ec1d: Status 404 returned error can't find the container with id 9539426ab59cbfc855a24fe93a71ddefdd0380d861881e7ac97d10340cc5ec1d
Jul 27 17:27:25.313491 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.313214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:25.313491 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.313368    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:25.313649 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.313512    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:26.313492216 +0000 UTC m=+4.024227861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:25.414496 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.414463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:25.414645 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.414519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:25.414713 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.414698    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:25.414771 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.414720    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:25.414771 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.414733    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:25.414866 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.414783    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:27:26.414766404 +0000 UTC m=+4.125502063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:25.415118 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.415093    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:25.415216 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.415151    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:27:26.415135958 +0000 UTC m=+4.125871618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:25.653030 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.652793    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:27:25.743227 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.743121    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:22:24 +0000 UTC" deadline="2028-02-01 16:18:40.529254419 +0000 UTC"
Jul 27 17:27:25.743227 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.743177    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13294h51m14.786081596s"
Jul 27 17:27:25.791765 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.791260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:25.791765 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:25.791411    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:25.802487 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.802457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jxl2n" event={"ID":"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e","Type":"ContainerStarted","Data":"bfee2dd2406a2d8ef26dfc99bfa15c93817bc8b39a123a75135ee883c2902547"}
Jul 27 17:27:25.805966 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.805937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r82m9" event={"ID":"141a7e0f-fdc5-4152-9626-c6387bece93a","Type":"ContainerStarted","Data":"ef5a3e094406ae075babece627bea08360e1fb264535f6ff6a2baefc30d7b2fd"}
Jul 27 17:27:25.813993 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.813957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerStarted","Data":"fcb6826b002531e743c3b4e43f22bc7ab7f453616c43002c3b74d3760d5ee67f"}
Jul 27 17:27:25.821361 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.821317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wk5wk" event={"ID":"56a1b52a-257c-48a7-8a73-e59e27e1a1d3","Type":"ContainerStarted","Data":"caf8a839d7829dfce585b498eb78246e55ea7590bc85ac0b938fa484d4ea3684"}
Jul 27 17:27:25.830411 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.830387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"9539426ab59cbfc855a24fe93a71ddefdd0380d861881e7ac97d10340cc5ec1d"}
Jul 27 17:27:25.834072 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.834047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-grtsm" event={"ID":"8193031b-4bdb-4756-9607-4fe816a48535","Type":"ContainerStarted","Data":"36cfd7499c7b3de675fb91b888c9351e07564a075a62015ea373db33176577a3"}
Jul 27 17:27:25.835503 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.835480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b4wnd" event={"ID":"1d4ac820-1268-4587-b1a8-fa9ad85ba912","Type":"ContainerStarted","Data":"aef98d3bf88d0c066103d460b87a90232ee914a38cf5c71e652d545ff771721b"}
Jul 27 17:27:25.842633 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.842608    2563 generic.go:356] "Generic (PLEG): container finished" podID="421b99f667c7b46ca0f75b4af30961f3" containerID="528ac700f085e4769db6ee0e81c18c5bcd99a6fc491f9410abab0886e2231ce5" exitCode=0
Jul 27 17:27:25.842729 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.842681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal" event={"ID":"421b99f667c7b46ca0f75b4af30961f3","Type":"ContainerDied","Data":"528ac700f085e4769db6ee0e81c18c5bcd99a6fc491f9410abab0886e2231ce5"}
Jul 27 17:27:25.844798 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.844772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx" event={"ID":"c74e1051-81e6-4b86-b04e-11df18125251","Type":"ContainerStarted","Data":"17a849b8b7ea08d843ef34e763d8d1e94a1b03a1e21e7d048d64d38b2b84cf91"}
Jul 27 17:27:25.846562 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:25.846540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjcvx" event={"ID":"78a9220d-c136-489e-84db-36da1c4423dc","Type":"ContainerStarted","Data":"96c16345e8b3e43dbc2dbdaaa87179a1d29bd5b10210fd34d4e04bd253fe570d"}
Jul 27 17:27:26.322504 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.322412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:26.322669 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.322551    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:26.322669 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.322624    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:28.322606353 +0000 UTC m=+6.033342011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.423362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.423419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.423613    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.423637    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.423676    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.423733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:27:28.423714412 +0000 UTC m=+6.134450064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.424115    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:26.424820 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.424161    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:27:28.424146439 +0000 UTC m=+6.134882090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:26.744248 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.744161    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:22:24 +0000 UTC" deadline="2028-03-19 19:33:39.66838021 +0000 UTC"
Jul 27 17:27:26.744248 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.744200    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14426h6m12.924184226s"
Jul 27 17:27:26.792570 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.792456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:26.792727 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.792576    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:26.792987 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.792824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:26.792987 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:26.792941    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:26.854520 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:26.854491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal" event={"ID":"421b99f667c7b46ca0f75b4af30961f3","Type":"ContainerStarted","Data":"04146e785ea87770c7be69ce8c4567258c49ad286cf4264d48f0aa0148bddc88"}
Jul 27 17:27:27.791701 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:27.791634    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:27.792168 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:27.791772    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:28.338667 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:28.338633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:28.338854 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.338790    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:28.338854 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.338853    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:32.338833848 +0000 UTC m=+10.049569506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:28.439620 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:28.439538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:28.439620 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:28.439590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:28.439834 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.439696    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:28.439834 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.439707    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:28.439834 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.439729    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:28.439834 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.439741    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:28.439834 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.439760    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:27:32.439740992 +0000 UTC m=+10.150476644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:28.439834 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.439789    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:27:32.439772975 +0000 UTC m=+10.150508636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:28.792203 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:28.791615    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:28.792203 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.791745    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:28.792953 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:28.792795    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:28.792953 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:28.792913    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:29.792074 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:29.792044    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:29.792260 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:29.792179    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:30.791754 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:30.791276    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:30.791754 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:30.791410    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:30.792130 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:30.792009    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:30.792130 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:30.792089    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:31.791389 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:31.791357    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:31.791839 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:31.791493    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:32.372192 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:32.371555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:32.372192 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.371709    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:32.372192 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.371775    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:40.371756247 +0000 UTC m=+18.082491895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:32.472384 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:32.472330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:32.472546 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:32.472406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:32.472616 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.472550    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:32.472616 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.472563    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:32.472616 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.472578    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:32.472616 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.472590    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:32.472616 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.472617    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:27:40.472598293 +0000 UTC m=+18.183333949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:32.472851 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.472637    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:27:40.472627191 +0000 UTC m=+18.183362836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:32.793259 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:32.792636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:32.793259 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.792739    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:32.793259 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:32.793070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:32.793259 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:32.793157    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:33.792152 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:33.792117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:33.792348 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:33.792253    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:34.791909 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:34.791874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:34.792376 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:34.792033    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:34.792376 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:34.792046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:34.792376 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:34.792185    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:35.791253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:35.791223    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:35.791420 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:35.791329    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:36.791715 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:36.791680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:36.792134 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:36.791815    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:36.792134 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:36.791871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:36.792134 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:36.791973    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:37.791855 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:37.791816    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:37.792267 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:37.791950    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:38.791885 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:38.791853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:38.792304 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:38.791853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:38.792304 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:38.791982    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:38.792304 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:38.792030    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:39.791928 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:39.791897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:39.792316 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:39.792029    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:40.429713 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:40.429679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:40.429875 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.429790    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:40.429875 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.429864    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:56.429828826 +0000 UTC m=+34.140564472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:40.530162 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:40.530133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:40.530321 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:40.530172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:40.530321 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.530292    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:40.530321 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.530303    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:40.530321 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.530316    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:40.530526 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.530326    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:40.530526 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.530371    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:27:56.530349728 +0000 UTC m=+34.241085394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:40.530526 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.530389    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:27:56.530381147 +0000 UTC m=+34.241116792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:40.791806 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:40.791773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:40.791962 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.791907    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:40.791962 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:40.791950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:40.792355 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:40.792053    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:41.791304 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:41.791270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:41.791498 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:41.791410    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:42.792512 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:42.792482    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:42.792910 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:42.792565    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:42.792910 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:42.792903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:42.793019 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:42.792992    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:43.791919 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:43.791888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:43.792089 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:43.792003    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:44.792136 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.791760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:44.792879 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.791760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:44.792879 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:44.792238    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:44.792879 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:44.792314    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:44.886757 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.886722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjcvx" event={"ID":"78a9220d-c136-489e-84db-36da1c4423dc","Type":"ContainerStarted","Data":"c33ae6fa35c2dbea7aea15bf9ce6bde703f086e0d6d69fb9c0f4de3d227065a0"}
Jul 27 17:27:44.887952 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.887926    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jxl2n" event={"ID":"e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e","Type":"ContainerStarted","Data":"03bc91678318a57f72667383fe9d47514e0fb12e7a2af34725c4c47ac5dc523d"}
Jul 27 17:27:44.889298 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.889280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r82m9" event={"ID":"141a7e0f-fdc5-4152-9626-c6387bece93a","Type":"ContainerStarted","Data":"29f8a320c11a8c630abcbeadc3fda1c83ce3917ef9d87088045021dc003f4384"}
Jul 27 17:27:44.890801 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.890782    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d69785c-df22-4d63-86f2-e60bfb5297f1" containerID="32c691a5f2105f521cbdccac2ada13df0e00e9f028fbfc58ec60d5cf103d26be" exitCode=0
Jul 27 17:27:44.890873 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.890837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerDied","Data":"32c691a5f2105f521cbdccac2ada13df0e00e9f028fbfc58ec60d5cf103d26be"}
Jul 27 17:27:44.892371 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.892324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wk5wk" event={"ID":"56a1b52a-257c-48a7-8a73-e59e27e1a1d3","Type":"ContainerStarted","Data":"dfee73035d4aceb64151009780f267794b8af3dc9dc52e8750f707c4c606981b"}
Jul 27 17:27:44.895226 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.895205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"6be082238cec0b2bd968aa67e9562cef5b42b3835b391536d7349e20e21a0023"}
Jul 27 17:27:44.895307 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.895261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"7b34e20d9f211806f30ec6bbf09960b5e4ce08b5208d2e309208103e2977bf72"}
Jul 27 17:27:44.895307 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.895295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"d947bc330cd2397af725aac0385150cdc473a5031ec9d06fa171f9ba1835c008"}
Jul 27 17:27:44.895408 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.895323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"02a1f3acbb8ab7209662f0ef470bc33076eccf690c2b57ee16ca604febc34cb4"}
Jul 27 17:27:44.895408 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.895368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"30cbdb6b91674361672f5d1398902edd4a7f54f28880b98b2359f327006d59bb"}
Jul 27 17:27:44.895408 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.895396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"f37c82232cce73a8d9ea9b8c1734029f530763cdfc6c082d4d717cec37bc1025"}
Jul 27 17:27:44.896565 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.896544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b4wnd" event={"ID":"1d4ac820-1268-4587-b1a8-fa9ad85ba912","Type":"ContainerStarted","Data":"f4ee20246d41f71de5ca8c704ad6c5b37688b762282f8945dccfe9b222b5948e"}
Jul 27 17:27:44.897831 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.897811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal" event={"ID":"1eab52b5abcc336338b0570279cda8a5","Type":"ContainerStarted","Data":"7b8f9f475b0426031c6e8302e73c20a9645b4456dc0c4b8280cea5ca0afdd33e"}
Jul 27 17:27:44.899440 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.899423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx" event={"ID":"c74e1051-81e6-4b86-b04e-11df18125251","Type":"ContainerStarted","Data":"bfb2142ccb7f345caf5a7c08df88e7e8c634c0458b4b5d70dc137b32ed4cc775"}
Jul 27 17:27:44.900946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.900907    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rjcvx" podStartSLOduration=3.089015892 podStartE2EDuration="21.900895862s" totalImagesPullingTime="18.81187997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:44.900271807 +0000 UTC m=+22.611007485" watchObservedRunningTime="2026-07-27 17:27:44.900895862 +0000 UTC m=+22.611631526"
Jul 27 17:27:44.901038 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.900976    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-235.ec2.internal" podStartSLOduration=20.900972607 podStartE2EDuration="20.900972607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:26.870532359 +0000 UTC m=+4.581268029" watchObservedRunningTime="2026-07-27 17:27:44.900972607 +0000 UTC m=+22.611708274"
Jul 27 17:27:44.912993 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.912958    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wk5wk" podStartSLOduration=11.881429155 podStartE2EDuration="22.912944271s" totalImagesPullingTime="11.031515116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:44.91290278 +0000 UTC m=+22.623638446" watchObservedRunningTime="2026-07-27 17:27:44.912944271 +0000 UTC m=+22.623679941"
Jul 27 17:27:44.927498 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.927463    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b4wnd" podStartSLOduration=4.093028208 podStartE2EDuration="22.92745229s" totalImagesPullingTime="18.834424082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:44.92702596 +0000 UTC m=+22.637761625" watchObservedRunningTime="2026-07-27 17:27:44.92745229 +0000 UTC m=+22.638187955"
Jul 27 17:27:44.942870 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.942841    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jxl2n" podStartSLOduration=3.125106619 podStartE2EDuration="21.94283076s" totalImagesPullingTime="18.817724141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:44.94276028 +0000 UTC m=+22.653495954" watchObservedRunningTime="2026-07-27 17:27:44.94283076 +0000 UTC m=+22.653566417"
Jul 27 17:27:44.962605 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:44.962573    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-235.ec2.internal" podStartSLOduration=20.962562427 podStartE2EDuration="20.962562427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:44.962383812 +0000 UTC m=+22.673119475" watchObservedRunningTime="2026-07-27 17:27:44.962562427 +0000 UTC m=+22.673298092"
Jul 27 17:27:45.364062 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.364032    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 17:27:45.755169 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.755052    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T17:27:45.364058415Z","UUID":"6feadf82-8f7e-410e-807b-21fba9cce72e","Handler":null,"Name":"","Endpoint":""}
Jul 27 17:27:45.756641 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.756610    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 17:27:45.756768 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.756650    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 17:27:45.791857 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.791833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:45.792065 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:45.791950    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:45.902591 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.902562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-grtsm" event={"ID":"8193031b-4bdb-4756-9607-4fe816a48535","Type":"ContainerStarted","Data":"5578f426519e39763e193dbfde3ea0af9b5658e0bf54d7476de85109a7fee9f3"}
Jul 27 17:27:45.904817 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.904784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx" event={"ID":"c74e1051-81e6-4b86-b04e-11df18125251","Type":"ContainerStarted","Data":"c2deca53921ec039e5637d6b21f27540ad33d612498bd9a58396eeae62533aed"}
Jul 27 17:27:45.916828 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.916789    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r82m9" podStartSLOduration=4.940794279 podStartE2EDuration="23.916777308s" totalImagesPullingTime="18.975983029s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:45.009373714 +0000 UTC m=+22.720109380" watchObservedRunningTime="2026-07-27 17:27:45.916777308 +0000 UTC m=+23.627512975"
Jul 27 17:27:45.916970 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:45.916931    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-grtsm" podStartSLOduration=4.122918795 podStartE2EDuration="22.916922849s" totalImagesPullingTime="18.794004054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:45.916400524 +0000 UTC m=+23.627136192" watchObservedRunningTime="2026-07-27 17:27:45.916922849 +0000 UTC m=+23.627658516"
Jul 27 17:27:46.481661 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:46.481631    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:46.482125 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:46.482108    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:46.791760 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:46.791569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:46.791936 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:46.791574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:46.791936 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:46.791852    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:46.792062 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:46.791981    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:46.912705 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:46.912204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"a04c654f597b20eca723946b32b90e904801678487b919700b745a25be558695"}
Jul 27 17:27:47.792101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:47.792069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:47.792290 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:47.792181    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:47.913967 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:47.913935    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 27 17:27:48.796925 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:48.796891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:48.796925 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:48.796927    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:48.797138 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:48.797002    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:48.797138 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:48.797112    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:48.866466 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:48.866438    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:48.867054 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:48.867034    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jxl2n"
Jul 27 17:27:49.792166 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.791992    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:49.792745 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:49.792245    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:49.920878 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.920847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx" event={"ID":"c74e1051-81e6-4b86-b04e-11df18125251","Type":"ContainerStarted","Data":"197e25ffd29048c314083a78ac0a73290a5861327abe0e0f75e1dc20417a3a9d"}
Jul 27 17:27:49.922561 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.922533    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d69785c-df22-4d63-86f2-e60bfb5297f1" containerID="7ad818f40caccaa0fc5c5dd78ed6f63722a565c62ae31c34011c1edcc9e93838" exitCode=0
Jul 27 17:27:49.922670 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.922613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerDied","Data":"7ad818f40caccaa0fc5c5dd78ed6f63722a565c62ae31c34011c1edcc9e93838"}
Jul 27 17:27:49.926183 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.926158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" event={"ID":"9066e682-b257-499f-8c53-b61cc205317b","Type":"ContainerStarted","Data":"7dcfa546e02deb3a6ebc39c2aa791af23b6d2b17e8b71553bbfc14fea74590c9"}
Jul 27 17:27:49.926560 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.926535    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:49.926643 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.926610    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:49.926696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.926653    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:49.938088 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.938034    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cdtjx" podStartSLOduration=3.285215811 podStartE2EDuration="26.938012513s" totalImagesPullingTime="23.652796702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:49.937883606 +0000 UTC m=+27.648619274" watchObservedRunningTime="2026-07-27 17:27:49.938012513 +0000 UTC m=+27.648748195"
Jul 27 17:27:49.942072 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.942052    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:49.942429 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.942411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:27:49.963643 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:49.963605    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr" podStartSLOduration=8.130112194 podStartE2EDuration="26.963591971s" totalImagesPullingTime="18.833479777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:27:49.963519053 +0000 UTC m=+27.674254719" watchObservedRunningTime="2026-07-27 17:27:49.963591971 +0000 UTC m=+27.674327668"
Jul 27 17:27:50.795295 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:50.795267    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:50.795802 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:50.795400    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:50.795802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:50.795447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:50.795802 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:50.795551    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:51.337604 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:51.337571    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-phrw4"]
Jul 27 17:27:51.337771 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:51.337675    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:51.337889 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:51.337765    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:51.340734 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:51.340712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n44f9"]
Jul 27 17:27:51.340858 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:51.340804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:51.340917 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:51.340887    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:51.342848 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:51.342823    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dxf4t"]
Jul 27 17:27:51.342931 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:51.342912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:51.343037 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:51.343003    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:52.794407 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:52.794183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:52.794799 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:52.794216    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:52.794799 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:52.794512    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:52.794799 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:52.794232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:52.794799 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:52.794581    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:52.794799 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:52.794629    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:52.934712 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:52.934682    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d69785c-df22-4d63-86f2-e60bfb5297f1" containerID="8e74fcff69a47772573f39b1407f18e21504ebfde098308fc404b9ad06e51e70" exitCode=0
Jul 27 17:27:52.934840 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:52.934730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerDied","Data":"8e74fcff69a47772573f39b1407f18e21504ebfde098308fc404b9ad06e51e70"}
Jul 27 17:27:54.794515 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:54.794489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:54.795185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:54.794587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:54.795185 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:54.794594    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:54.795185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:54.794612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:54.795185 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:54.794684    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:54.795185 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:54.794745    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:55.942681 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:55.942651    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d69785c-df22-4d63-86f2-e60bfb5297f1" containerID="27938f0ff3e2f6b6673bd70f2f4c437f473de14f1af1992edf1d19d332aa8c52" exitCode=0
Jul 27 17:27:55.943133 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:55.942706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerDied","Data":"27938f0ff3e2f6b6673bd70f2f4c437f473de14f1af1992edf1d19d332aa8c52"}
Jul 27 17:27:56.455306 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:56.455265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:56.455498 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.455466    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:56.455566 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.455537    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret podName:04a2888d-ead2-4657-a191-b7fe4d26c689 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:28.455515211 +0000 UTC m=+66.166250872 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret") pod "global-pull-secret-syncer-phrw4" (UID: "04a2888d-ead2-4657-a191-b7fe4d26c689") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:27:56.555701 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:56.555667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:56.555701 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:56.555702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:56.555904 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.555795    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:56.555904 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.555814    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:27:56.555904 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.555830    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:27:56.555904 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.555841    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzpn2 for pod openshift-network-diagnostics/network-check-target-dxf4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:56.555904 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.555859    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:28:28.555838539 +0000 UTC m=+66.266574183 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:27:56.555904 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.555883    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2 podName:2e5f8925-e357-4ef5-9b0b-1644bfe959ff nodeName:}" failed. No retries permitted until 2026-07-27 17:28:28.555867368 +0000 UTC m=+66.266603028 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzpn2" (UniqueName: "kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2") pod "network-check-target-dxf4t" (UID: "2e5f8925-e357-4ef5-9b0b-1644bfe959ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:27:56.791789 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:56.791761    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:56.792022 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:56.791770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:56.792022 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.791897    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n44f9" podUID="269531c4-3fdf-4ceb-8046-2228ac3c143d"
Jul 27 17:27:56.792022 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:56.791769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:56.792022 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.791987    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phrw4" podUID="04a2888d-ead2-4657-a191-b7fe4d26c689"
Jul 27 17:27:56.792226 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:56.792028    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dxf4t" podUID="2e5f8925-e357-4ef5-9b0b-1644bfe959ff"
Jul 27 17:27:57.571606 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.571575    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-235.ec2.internal" event="NodeReady"
Jul 27 17:27:57.572091 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.571723    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 17:27:57.606923 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.606573    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-88f4bc747-qlckf"]
Jul 27 17:27:57.622929 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.622890    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sk7vl"]
Jul 27 17:27:57.623107 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.623047    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.625612 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.625575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 17:27:57.625612 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.625601    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 17:27:57.625612 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.625608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lcj8p\""
Jul 27 17:27:57.625831 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.625617    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 17:27:57.631577 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.631552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 17:27:57.639216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.639196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-88f4bc747-qlckf"]
Jul 27 17:27:57.639216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.639233    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dkhl7"]
Jul 27 17:27:57.639447 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.639359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.642889 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.642874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 17:27:57.643120 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.643106    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 17:27:57.643205 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.643123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x9hw5\""
Jul 27 17:27:57.657516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.657462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sk7vl"]
Jul 27 17:27:57.657516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.657497    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dkhl7"]
Jul 27 17:27:57.657665 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.657594    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.659624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.659604    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 17:27:57.659722 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.659653    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 17:27:57.659722 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.659670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c5l2l\""
Jul 27 17:27:57.663188 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.663154    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 17:27:57.765209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.765410 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-bound-sa-token\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.765410 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d90298b-c573-4d45-accc-086b982b6d8e-cert\") pod \"ingress-canary-dkhl7\" (UID: \"6d90298b-c573-4d45-accc-086b982b6d8e\") " pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.765600 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765442    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-image-registry-private-configuration\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.765791 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90e620ae-b8f6-4ccd-b01c-19b402549622-ca-trust-extracted\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.765874 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-installation-pull-secrets\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.765874 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.765968 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/25394289-0555-486d-a7a4-bf37a57caf8a-config-volume\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.765968 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68dlh\" (UniqueName: \"kubernetes.io/projected/6d90298b-c573-4d45-accc-086b982b6d8e-kube-api-access-68dlh\") pod \"ingress-canary-dkhl7\" (UID: \"6d90298b-c573-4d45-accc-086b982b6d8e\") " pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.766042 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.765978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25394289-0555-486d-a7a4-bf37a57caf8a-tmp-dir\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.766042 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.766011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkw65\" (UniqueName: \"kubernetes.io/projected/25394289-0555-486d-a7a4-bf37a57caf8a-kube-api-access-fkw65\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.766102 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.766041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd7xl\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-kube-api-access-zd7xl\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.766149 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.766098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-trusted-ca\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.766188 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.766177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-certificates\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866692 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-trusted-ca\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-certificates\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-bound-sa-token\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d90298b-c573-4d45-accc-086b982b6d8e-cert\") pod \"ingress-canary-dkhl7\" (UID: \"6d90298b-c573-4d45-accc-086b982b6d8e\") " pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-image-registry-private-configuration\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90e620ae-b8f6-4ccd-b01c-19b402549622-ca-trust-extracted\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-installation-pull-secrets\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.866946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/25394289-0555-486d-a7a4-bf37a57caf8a-config-volume\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:57.866975    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:57.866998    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:57.867061    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:58.367039703 +0000 UTC m=+36.077775367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.866983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68dlh\" (UniqueName: \"kubernetes.io/projected/6d90298b-c573-4d45-accc-086b982b6d8e-kube-api-access-68dlh\") pod \"ingress-canary-dkhl7\" (UID: \"6d90298b-c573-4d45-accc-086b982b6d8e\") " pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25394289-0555-486d-a7a4-bf37a57caf8a-tmp-dir\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkw65\" (UniqueName: \"kubernetes.io/projected/25394289-0555-486d-a7a4-bf37a57caf8a-kube-api-access-fkw65\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.867324 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd7xl\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-kube-api-access-zd7xl\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.867736 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90e620ae-b8f6-4ccd-b01c-19b402549622-ca-trust-extracted\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.867736 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25394289-0555-486d-a7a4-bf37a57caf8a-tmp-dir\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.867736 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-certificates\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.867736 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:57.867647    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:27:57.867907 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-trusted-ca\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.867907 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:57.867755    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:27:58.367737216 +0000 UTC m=+36.078472874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:27:57.868011 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.867986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/25394289-0555-486d-a7a4-bf37a57caf8a-config-volume\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.871820 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.871688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-installation-pull-secrets\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.871820 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.871704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d90298b-c573-4d45-accc-086b982b6d8e-cert\") pod \"ingress-canary-dkhl7\" (UID: \"6d90298b-c573-4d45-accc-086b982b6d8e\") " pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.871986 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.871803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-image-registry-private-configuration\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.878260 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.878234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-bound-sa-token\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.878445 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.878310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkw65\" (UniqueName: \"kubernetes.io/projected/25394289-0555-486d-a7a4-bf37a57caf8a-kube-api-access-fkw65\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:57.878445 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.878410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68dlh\" (UniqueName: \"kubernetes.io/projected/6d90298b-c573-4d45-accc-086b982b6d8e-kube-api-access-68dlh\") pod \"ingress-canary-dkhl7\" (UID: \"6d90298b-c573-4d45-accc-086b982b6d8e\") " pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:57.878529 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.878465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd7xl\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-kube-api-access-zd7xl\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:57.966759 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:57.966671    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dkhl7"
Jul 27 17:27:58.148386 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.148307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dkhl7"]
Jul 27 17:27:58.152721 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:27:58.152693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d90298b_c573_4d45_accc_086b982b6d8e.slice/crio-9ceb80691bc312ffd4946a832a13ad745a8e48dcb0b60b030f24dc67cb19b2b4 WatchSource:0}: Error finding container 9ceb80691bc312ffd4946a832a13ad745a8e48dcb0b60b030f24dc67cb19b2b4: Status 404 returned error can't find the container with id 9ceb80691bc312ffd4946a832a13ad745a8e48dcb0b60b030f24dc67cb19b2b4
Jul 27 17:27:58.370571 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.370542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:58.370571 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.370577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:58.370884 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:58.370682    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:27:58.370884 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:58.370702    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:27:58.370884 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:58.370724    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:27:58.370884 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:58.370738    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:27:59.370724985 +0000 UTC m=+37.081460630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:27:58.370884 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:58.370778    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:27:59.370760819 +0000 UTC m=+37.081496481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:27:58.795545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.795516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:27:58.796014 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.795524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:27:58.796014 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.795660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:27:58.798310 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.798289    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:27:58.798455 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.798402    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:27:58.799225 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.799206    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6sz2v\""
Jul 27 17:27:58.799225 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.799220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcn4g\""
Jul 27 17:27:58.799401 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.799262    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:27:58.799401 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.799284    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:27:58.950925 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:58.950878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dkhl7" event={"ID":"6d90298b-c573-4d45-accc-086b982b6d8e","Type":"ContainerStarted","Data":"9ceb80691bc312ffd4946a832a13ad745a8e48dcb0b60b030f24dc67cb19b2b4"}
Jul 27 17:27:59.379823 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:59.379789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:27:59.380016 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:27:59.379833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:27:59.380016 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:59.379948    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:27:59.380016 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:59.379964    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:27:59.380016 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:59.379985    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:27:59.380192 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:59.380045    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:01.380024176 +0000 UTC m=+39.090759823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:27:59.380192 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:27:59.380066    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:01.380056053 +0000 UTC m=+39.090791700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:28:00.958905 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:00.958668    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d69785c-df22-4d63-86f2-e60bfb5297f1" containerID="b4d9d7a402a76d45dd22d646b41c0c01f93f586172cd67f324db211795930b09" exitCode=0
Jul 27 17:28:00.959305 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:00.958935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerDied","Data":"b4d9d7a402a76d45dd22d646b41c0c01f93f586172cd67f324db211795930b09"}
Jul 27 17:28:01.298566 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:01.298526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:01.398178 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:01.398146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:28:01.398178 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:01.398183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:28:01.398408 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:01.398301    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:28:01.398408 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:01.398308    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:28:01.398408 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:01.398327    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:28:01.398408 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:01.398371    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:05.398350894 +0000 UTC m=+43.109086563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:28:01.398408 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:01.398393    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:05.398379929 +0000 UTC m=+43.109115573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:28:01.963318 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:01.963282    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d69785c-df22-4d63-86f2-e60bfb5297f1" containerID="f67447b306adcc4d63a859f1db5defdcc03b065fcccedbc5da93c9926e3aec4a" exitCode=0
Jul 27 17:28:01.963709 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:01.963371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerDied","Data":"f67447b306adcc4d63a859f1db5defdcc03b065fcccedbc5da93c9926e3aec4a"}
Jul 27 17:28:02.967146 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:02.967111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dkhl7" event={"ID":"6d90298b-c573-4d45-accc-086b982b6d8e","Type":"ContainerStarted","Data":"f8ac6231dec0e62fd3d89bf2410f55e17b12b45dcc1e251b77cc3c0e082c1fa8"}
Jul 27 17:28:02.970296 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:02.970270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" event={"ID":"4d69785c-df22-4d63-86f2-e60bfb5297f1","Type":"ContainerStarted","Data":"99bdf748b6ecf7f453510ee16537d34ace2c84d568dcad22efdfdb02cd4fa056"}
Jul 27 17:28:02.981672 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:02.981626    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dkhl7" podStartSLOduration=1.980970069 podStartE2EDuration="5.981612953s" totalImagesPullingTime="4.000642884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:02.980972806 +0000 UTC m=+40.691708474" watchObservedRunningTime="2026-07-27 17:28:02.981612953 +0000 UTC m=+40.692348620"
Jul 27 17:28:03.000276 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:03.000231    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hzgdr" podStartSLOduration=9.717413586 podStartE2EDuration="41.000215767s" totalImagesPullingTime="31.282802181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:02.99960787 +0000 UTC m=+40.710343547" watchObservedRunningTime="2026-07-27 17:28:03.000215767 +0000 UTC m=+40.710951439"
Jul 27 17:28:03.121147 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:03.121121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:03.268166 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:03.268104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:03.384262 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:03.384232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:03.479799 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:03.479771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:03.573939 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:03.573911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:04.065978 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:04.065951    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:04.157362 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:04.157321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:04.717505 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:04.717470    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:04.817442 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:04.817407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:05.283879 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:05.283838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:05.384650 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:05.384617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:05.426669 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:05.426643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:28:05.426807 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:05.426731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:28:05.426868 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:05.426805    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:28:05.426868 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:05.426822    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:28:05.426868 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:05.426833    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:28:05.426998 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:05.426880    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:13.426857502 +0000 UTC m=+51.137593165 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:28:05.426998 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:05.426900    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:13.42688995 +0000 UTC m=+51.137625600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:28:05.671845 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:05.671761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:05.724219 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:05.724187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:06.472499 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:06.472466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:06.528893 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:06.528866    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:06.869786 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:06.869745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:06.969330 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:06.969297    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:13.484568 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:13.484524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:28:13.484568 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:13.484576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:28:13.485070 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:13.484686    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:28:13.485070 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:13.484703    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:28:13.485070 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:13.484716    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:28:13.485070 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:13.484762    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:29.484747211 +0000 UTC m=+67.195482861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:28:13.485070 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:13.484777    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:29.484772159 +0000 UTC m=+67.195507804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:28:21.940939 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:21.940910    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxgdr"
Jul 27 17:28:28.482622 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.482583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:28:28.485266 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.485248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:28:28.496250 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.496222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04a2888d-ead2-4657-a191-b7fe4d26c689-original-pull-secret\") pod \"global-pull-secret-syncer-phrw4\" (UID: \"04a2888d-ead2-4657-a191-b7fe4d26c689\") " pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:28:28.513405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.513383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phrw4"
Jul 27 17:28:28.583595 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.583512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:28:28.583738 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.583639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:28:28.586209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.585973    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:28:28.586209 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.586175    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:28:28.593656 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:28.593634    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 17:28:28.593760 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:28.593709    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs podName:269531c4-3fdf-4ceb-8046-2228ac3c143d nodeName:}" failed. No retries permitted until 2026-07-27 17:29:32.593693121 +0000 UTC m=+130.304428770 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs") pod "network-metrics-daemon-n44f9" (UID: "269531c4-3fdf-4ceb-8046-2228ac3c143d") : secret "metrics-daemon-secret" not found
Jul 27 17:28:28.596774 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.596752    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:28:28.607548 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.607522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzpn2\" (UniqueName: \"kubernetes.io/projected/2e5f8925-e357-4ef5-9b0b-1644bfe959ff-kube-api-access-tzpn2\") pod \"network-check-target-dxf4t\" (UID: \"2e5f8925-e357-4ef5-9b0b-1644bfe959ff\") " pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:28:28.629426 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.629379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-phrw4"]
Jul 27 17:28:28.632797 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:28.632773    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a2888d_ead2_4657_a191_b7fe4d26c689.slice/crio-66c6f287d095fe82fd011bfd36fc7d5ac9eb3f0e0c89e44f5320faa1c6622e0f WatchSource:0}: Error finding container 66c6f287d095fe82fd011bfd36fc7d5ac9eb3f0e0c89e44f5320faa1c6622e0f: Status 404 returned error can't find the container with id 66c6f287d095fe82fd011bfd36fc7d5ac9eb3f0e0c89e44f5320faa1c6622e0f
Jul 27 17:28:28.809974 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.809951    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcn4g\""
Jul 27 17:28:28.817660 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.817629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:28:28.942587 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:28.942561    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dxf4t"]
Jul 27 17:28:28.945783 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:28.945756    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5f8925_e357_4ef5_9b0b_1644bfe959ff.slice/crio-4a02984f6b50cc292744a7e1581bf7b2b85248b59939a5e43ecd78e6731502cd WatchSource:0}: Error finding container 4a02984f6b50cc292744a7e1581bf7b2b85248b59939a5e43ecd78e6731502cd: Status 404 returned error can't find the container with id 4a02984f6b50cc292744a7e1581bf7b2b85248b59939a5e43ecd78e6731502cd
Jul 27 17:28:29.028241 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:29.028215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dxf4t" event={"ID":"2e5f8925-e357-4ef5-9b0b-1644bfe959ff","Type":"ContainerStarted","Data":"4a02984f6b50cc292744a7e1581bf7b2b85248b59939a5e43ecd78e6731502cd"}
Jul 27 17:28:29.029217 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:29.029192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-phrw4" event={"ID":"04a2888d-ead2-4657-a191-b7fe4d26c689","Type":"ContainerStarted","Data":"66c6f287d095fe82fd011bfd36fc7d5ac9eb3f0e0c89e44f5320faa1c6622e0f"}
Jul 27 17:28:29.489557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:29.489519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:28:29.489969 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:29.489579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:28:29.489969 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:29.489782    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:28:29.489969 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:29.489830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls podName:25394289-0555-486d-a7a4-bf37a57caf8a nodeName:}" failed. No retries permitted until 2026-07-27 17:29:01.489816357 +0000 UTC m=+99.200552002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls") pod "dns-default-sk7vl" (UID: "25394289-0555-486d-a7a4-bf37a57caf8a") : secret "dns-default-metrics-tls" not found
Jul 27 17:28:29.489969 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:29.489897    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:28:29.489969 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:29.489905    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88f4bc747-qlckf: secret "image-registry-tls" not found
Jul 27 17:28:29.489969 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:29.489928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls podName:90e620ae-b8f6-4ccd-b01c-19b402549622 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:01.489920836 +0000 UTC m=+99.200656480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls") pod "image-registry-88f4bc747-qlckf" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622") : secret "image-registry-tls" not found
Jul 27 17:28:34.044315 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.044280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dxf4t" event={"ID":"2e5f8925-e357-4ef5-9b0b-1644bfe959ff","Type":"ContainerStarted","Data":"cf534fb01509f946377dbed41c2e11736c415987d29607c27ddbbdd5351d0770"}
Jul 27 17:28:34.044751 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.044391    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:28:34.045834 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.045802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-phrw4" event={"ID":"04a2888d-ead2-4657-a191-b7fe4d26c689","Type":"ContainerStarted","Data":"729dd74559e6d44594ae98067a59deae083cb5870aa03b48e4c1ecb1be4b93b2"}
Jul 27 17:28:34.065737 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.065690    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dxf4t" podStartSLOduration=67.806694888 podStartE2EDuration="1m12.065675462s" totalImagesPullingTime="4.258980574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:34.062701513 +0000 UTC m=+71.773437177" watchObservedRunningTime="2026-07-27 17:28:34.065675462 +0000 UTC m=+71.776411109"
Jul 27 17:28:34.066241 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.066224    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"]
Jul 27 17:28:34.068951 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.068937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.070466 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.070435    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 17:28:34.070718 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.070699    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 17:28:34.070871 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.070855    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.070938 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.070919    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-j4vvv\""
Jul 27 17:28:34.071000 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.070923    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.075835 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.075815    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"]
Jul 27 17:28:34.092258 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.092201    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-phrw4" podStartSLOduration=66.515719173 podStartE2EDuration="1m11.092183804s" totalImagesPullingTime="4.576464631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:34.078811716 +0000 UTC m=+71.789547382" watchObservedRunningTime="2026-07-27 17:28:34.092183804 +0000 UTC m=+71.802919472"
Jul 27 17:28:34.126148 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.126121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-serving-cert\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.126384 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.126364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm7k6\" (UniqueName: \"kubernetes.io/projected/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-kube-api-access-gm7k6\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.126449 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.126409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-config\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.168135 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.168108    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"]
Jul 27 17:28:34.170856 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.170843    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"
Jul 27 17:28:34.172862 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.172842    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.172955 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.172938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.173066 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.173048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vfg8b\""
Jul 27 17:28:34.174686 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.174666    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"]
Jul 27 17:28:34.177566 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.177545    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jwmmm"]
Jul 27 17:28:34.177683 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.177664    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.179629 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.179609    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jncw4\""
Jul 27 17:28:34.179741 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.179642    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.180211 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.180192    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"]
Jul 27 17:28:34.180353 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.180323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.180658 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.180494    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.180658 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.180533    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 17:28:34.180809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.180791    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 17:28:34.182113 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.182095    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tm5bt\""
Jul 27 17:28:34.182113 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.182108    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.182408 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.182380    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.182491 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.182404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 17:28:34.182657 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.182638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 17:28:34.182977 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.182963    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"]
Jul 27 17:28:34.183254 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.183084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.185240 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.185221    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 17:28:34.185357 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.185220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 17:28:34.185357 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.185224    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.185861 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.185839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"]
Jul 27 17:28:34.185976 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.185927    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.187867 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.187364    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.187867 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.187491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-l7j7m\""
Jul 27 17:28:34.189028 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.189001    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"]
Jul 27 17:28:34.189292 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.189273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 17:28:34.190143 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.190125    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.190622 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.190598    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.190886 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.190865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bx2xk\""
Jul 27 17:28:34.193680 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.193622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jwmmm"]
Jul 27 17:28:34.194627 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.194607    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"]
Jul 27 17:28:34.194855 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.194838    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 17:28:34.203729 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.203707    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"]
Jul 27 17:28:34.226652 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d696cb3d-caf7-4bfe-876d-eca689f294f0-tmp\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.226776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d696cb3d-caf7-4bfe-876d-eca689f294f0-service-ca-bundle\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.226776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhx47\" (UniqueName: \"kubernetes.io/projected/d696cb3d-caf7-4bfe-876d-eca689f294f0-kube-api-access-jhx47\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.226776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-serving-cert\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.226776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bkj7\" (UniqueName: \"kubernetes.io/projected/91c5285c-1b1f-4512-874a-5d4fc63cb925-kube-api-access-6bkj7\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.226776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld5lw\" (UniqueName: \"kubernetes.io/projected/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-kube-api-access-ld5lw\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm7k6\" (UniqueName: \"kubernetes.io/projected/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-kube-api-access-gm7k6\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs27q\" (UniqueName: \"kubernetes.io/projected/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-kube-api-access-hs27q\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d696cb3d-caf7-4bfe-876d-eca689f294f0-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d696cb3d-caf7-4bfe-876d-eca689f294f0-serving-cert\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91c5285c-1b1f-4512-874a-5d4fc63cb925-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.226996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.227039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.227030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.227347 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.227118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc85b\" (UniqueName: \"kubernetes.io/projected/9f754619-6367-4fd6-ab8a-4c9367f5bb6f-kube-api-access-lc85b\") pod \"volume-data-source-validator-549c6d54cf-lflhc\" (UID: \"9f754619-6367-4fd6-ab8a-4c9367f5bb6f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"
Jul 27 17:28:34.227347 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.227142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d696cb3d-caf7-4bfe-876d-eca689f294f0-snapshots\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.227347 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.227160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91c5285c-1b1f-4512-874a-5d4fc63cb925-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.227347 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.227196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-config\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.227705 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.227688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-config\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.228921 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.228902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-serving-cert\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.234082 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.234061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm7k6\" (UniqueName: \"kubernetes.io/projected/9850ae52-17e3-42bb-a0c7-c73f28f71e8d-kube-api-access-gm7k6\") pod \"service-ca-operator-5d6c75f84f-5mx9w\" (UID: \"9850ae52-17e3-42bb-a0c7-c73f28f71e8d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.274551 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.274527    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fxvl2"]
Jul 27 17:28:34.277607 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.277593    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.280095 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.280075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.280202 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.280113    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 17:28:34.280202 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.280179    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 17:28:34.280472 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.280456    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.280621 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.280508    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vfp88\""
Jul 27 17:28:34.287119 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.287100    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 17:28:34.290724 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.290704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fxvl2"]
Jul 27 17:28:34.291539 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.291518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6dbb545db7-fnmlq"]
Jul 27 17:28:34.294803 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.294758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.296704 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.296683    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 17:28:34.296704 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.296695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 17:28:34.296838 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.296728    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 17:28:34.296838 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.296780    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sf5qf\""
Jul 27 17:28:34.296838 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.296803    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 17:28:34.296838 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.296830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 17:28:34.297045 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.297032    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 17:28:34.309522 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.309502    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dbb545db7-fnmlq"]
Jul 27 17:28:34.328287 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc85b\" (UniqueName: \"kubernetes.io/projected/9f754619-6367-4fd6-ab8a-4c9367f5bb6f-kube-api-access-lc85b\") pod \"volume-data-source-validator-549c6d54cf-lflhc\" (UID: \"9f754619-6367-4fd6-ab8a-4c9367f5bb6f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"
Jul 27 17:28:34.328405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d696cb3d-caf7-4bfe-876d-eca689f294f0-snapshots\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.328405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91c5285c-1b1f-4512-874a-5d4fc63cb925-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.328405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.328545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4417e95-0dab-43b2-aea7-e774b713f5fe-config\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.328545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d696cb3d-caf7-4bfe-876d-eca689f294f0-tmp\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.328545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d696cb3d-caf7-4bfe-876d-eca689f294f0-service-ca-bundle\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.328545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhx47\" (UniqueName: \"kubernetes.io/projected/d696cb3d-caf7-4bfe-876d-eca689f294f0-kube-api-access-jhx47\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.328545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bkj7\" (UniqueName: \"kubernetes.io/projected/91c5285c-1b1f-4512-874a-5d4fc63cb925-kube-api-access-6bkj7\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.328776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld5lw\" (UniqueName: \"kubernetes.io/projected/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-kube-api-access-ld5lw\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.328776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4417e95-0dab-43b2-aea7-e774b713f5fe-serving-cert\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.328776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-stats-auth\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.328776 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4417e95-0dab-43b2-aea7-e774b713f5fe-trusted-ca\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.328972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs27q\" (UniqueName: \"kubernetes.io/projected/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-kube-api-access-hs27q\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.328972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328868    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4z29\" (UniqueName: \"kubernetes.io/projected/f4417e95-0dab-43b2-aea7-e774b713f5fe-kube-api-access-t4z29\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.328972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d696cb3d-caf7-4bfe-876d-eca689f294f0-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.328972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d696cb3d-caf7-4bfe-876d-eca689f294f0-serving-cert\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.328972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91c5285c-1b1f-4512-874a-5d4fc63cb925-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.328988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d696cb3d-caf7-4bfe-876d-eca689f294f0-snapshots\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d696cb3d-caf7-4bfe-876d-eca689f294f0-service-ca-bundle\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gjrw\" (UniqueName: \"kubernetes.io/projected/0e810fcc-7e49-4772-9ade-c9731493e4f7-kube-api-access-5gjrw\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-default-certificate\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.329144    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:28:34.329280 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.329218    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:34.329753 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.329291    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls podName:ed0673a0-b03c-4ac0-8c34-08eb4a491e2a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:34.82927114 +0000 UTC m=+72.540006789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls") pod "cluster-samples-operator-6748489656-2qggm" (UID: "ed0673a0-b03c-4ac0-8c34-08eb4a491e2a") : secret "samples-operator-tls" not found
Jul 27 17:28:34.329753 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d696cb3d-caf7-4bfe-876d-eca689f294f0-tmp\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.329753 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.329352    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls podName:80a87ec3-31ec-4ddb-ae24-b9ef51e9c227 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:34.829318494 +0000 UTC m=+72.540054152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-xbnbm" (UID: "80a87ec3-31ec-4ddb-ae24-b9ef51e9c227") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:34.329753 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329685    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91c5285c-1b1f-4512-874a-5d4fc63cb925-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.329903 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.329864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.330295 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.330275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d696cb3d-caf7-4bfe-876d-eca689f294f0-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.331069 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.331049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91c5285c-1b1f-4512-874a-5d4fc63cb925-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.331210 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.331194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d696cb3d-caf7-4bfe-876d-eca689f294f0-serving-cert\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.339234 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.339195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc85b\" (UniqueName: \"kubernetes.io/projected/9f754619-6367-4fd6-ab8a-4c9367f5bb6f-kube-api-access-lc85b\") pod \"volume-data-source-validator-549c6d54cf-lflhc\" (UID: \"9f754619-6367-4fd6-ab8a-4c9367f5bb6f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"
Jul 27 17:28:34.339522 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.339496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bkj7\" (UniqueName: \"kubernetes.io/projected/91c5285c-1b1f-4512-874a-5d4fc63cb925-kube-api-access-6bkj7\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv\" (UID: \"91c5285c-1b1f-4512-874a-5d4fc63cb925\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.339522 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.339510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs27q\" (UniqueName: \"kubernetes.io/projected/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-kube-api-access-hs27q\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.339979 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.339958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld5lw\" (UniqueName: \"kubernetes.io/projected/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-kube-api-access-ld5lw\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.340077 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.339967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhx47\" (UniqueName: \"kubernetes.io/projected/d696cb3d-caf7-4bfe-876d-eca689f294f0-kube-api-access-jhx47\") pod \"insights-operator-5885fdb759-jwmmm\" (UID: \"d696cb3d-caf7-4bfe-876d-eca689f294f0\") " pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.377820 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.377795    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"
Jul 27 17:28:34.430290 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4z29\" (UniqueName: \"kubernetes.io/projected/f4417e95-0dab-43b2-aea7-e774b713f5fe-kube-api-access-t4z29\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.430453 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.430517 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gjrw\" (UniqueName: \"kubernetes.io/projected/0e810fcc-7e49-4772-9ade-c9731493e4f7-kube-api-access-5gjrw\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.430571 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-default-certificate\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.430628 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.430676 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4417e95-0dab-43b2-aea7-e774b713f5fe-config\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.430676 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.430650    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:28:34.430810 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4417e95-0dab-43b2-aea7-e774b713f5fe-serving-cert\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.430810 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-stats-auth\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.430810 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.430728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4417e95-0dab-43b2-aea7-e774b713f5fe-trusted-ca\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.430810 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.430806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:34.930783286 +0000 UTC m=+72.641518937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : secret "router-metrics-certs-default" not found
Jul 27 17:28:34.431184 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.431145    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:34.931128323 +0000 UTC m=+72.641863999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : configmap references non-existent config key: service-ca.crt
Jul 27 17:28:34.431496 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.431460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4417e95-0dab-43b2-aea7-e774b713f5fe-config\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.432187 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.432160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4417e95-0dab-43b2-aea7-e774b713f5fe-trusted-ca\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.433734 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.433705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4417e95-0dab-43b2-aea7-e774b713f5fe-serving-cert\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.434426 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.434406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-stats-auth\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.434685 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.434660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-default-certificate\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.442106 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.442082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4z29\" (UniqueName: \"kubernetes.io/projected/f4417e95-0dab-43b2-aea7-e774b713f5fe-kube-api-access-t4z29\") pod \"console-operator-584dcb6b54-fxvl2\" (UID: \"f4417e95-0dab-43b2-aea7-e774b713f5fe\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.443556 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.443155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gjrw\" (UniqueName: \"kubernetes.io/projected/0e810fcc-7e49-4772-9ade-c9731493e4f7-kube-api-access-5gjrw\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.479959 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.479932    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"
Jul 27 17:28:34.499525 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.499428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w"]
Jul 27 17:28:34.501467 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.501400    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-jwmmm"
Jul 27 17:28:34.502712 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:34.502689    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9850ae52_17e3_42bb_a0c7_c73f28f71e8d.slice/crio-779bd633c3b905ccfbba7ff797da9db35104730fb20e9e861af489db38c40484 WatchSource:0}: Error finding container 779bd633c3b905ccfbba7ff797da9db35104730fb20e9e861af489db38c40484: Status 404 returned error can't find the container with id 779bd633c3b905ccfbba7ff797da9db35104730fb20e9e861af489db38c40484
Jul 27 17:28:34.507498 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.507475    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"
Jul 27 17:28:34.587131 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.587079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:34.624010 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.623937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc"]
Jul 27 17:28:34.628393 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:34.628367    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f754619_6367_4fd6_ab8a_4c9367f5bb6f.slice/crio-960d2f604317ae2ce97d306d191b0e998f3494971daf78a1e6eb0a65a7dc57f5 WatchSource:0}: Error finding container 960d2f604317ae2ce97d306d191b0e998f3494971daf78a1e6eb0a65a7dc57f5: Status 404 returned error can't find the container with id 960d2f604317ae2ce97d306d191b0e998f3494971daf78a1e6eb0a65a7dc57f5
Jul 27 17:28:34.643513 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.643481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jwmmm"]
Jul 27 17:28:34.650292 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:34.650263    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd696cb3d_caf7_4bfe_876d_eca689f294f0.slice/crio-c9ca005173eb7e56767e315e3bbe4ad7749477f980fc399274a5b2fd85ed6c9f WatchSource:0}: Error finding container c9ca005173eb7e56767e315e3bbe4ad7749477f980fc399274a5b2fd85ed6c9f: Status 404 returned error can't find the container with id c9ca005173eb7e56767e315e3bbe4ad7749477f980fc399274a5b2fd85ed6c9f
Jul 27 17:28:34.666570 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.666546    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv"]
Jul 27 17:28:34.670740 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:34.670709    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c5285c_1b1f_4512_874a_5d4fc63cb925.slice/crio-97315accbf45e44d5284c0f3a4c6b43a7be84f738418d29802966781e9d7c225 WatchSource:0}: Error finding container 97315accbf45e44d5284c0f3a4c6b43a7be84f738418d29802966781e9d7c225: Status 404 returned error can't find the container with id 97315accbf45e44d5284c0f3a4c6b43a7be84f738418d29802966781e9d7c225
Jul 27 17:28:34.720175 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.720149    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fxvl2"]
Jul 27 17:28:34.723036 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:34.723009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4417e95_0dab_43b2_aea7_e774b713f5fe.slice/crio-319cc39461f1ec762d65e0b57058c7edbc11659f89b13c4026a460f091fada18 WatchSource:0}: Error finding container 319cc39461f1ec762d65e0b57058c7edbc11659f89b13c4026a460f091fada18: Status 404 returned error can't find the container with id 319cc39461f1ec762d65e0b57058c7edbc11659f89b13c4026a460f091fada18
Jul 27 17:28:34.833366 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.833265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:34.833366 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.833299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:34.833543 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.833424    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:34.833543 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.833473    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls podName:80a87ec3-31ec-4ddb-ae24-b9ef51e9c227 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:35.83346018 +0000 UTC m=+73.544195826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-xbnbm" (UID: "80a87ec3-31ec-4ddb-ae24-b9ef51e9c227") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:34.833543 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.833424    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:28:34.833664 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.833556    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls podName:ed0673a0-b03c-4ac0-8c34-08eb4a491e2a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:35.833542852 +0000 UTC m=+73.544278502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls") pod "cluster-samples-operator-6748489656-2qggm" (UID: "ed0673a0-b03c-4ac0-8c34-08eb4a491e2a") : secret "samples-operator-tls" not found
Jul 27 17:28:34.933995 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.933967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.934148 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:34.934054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:34.934148 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.934101    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:28:34.934225 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.934160    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:35.934145917 +0000 UTC m=+73.644881563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : secret "router-metrics-certs-default" not found
Jul 27 17:28:34.934225 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:34.934176    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:35.934170133 +0000 UTC m=+73.644905807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : configmap references non-existent config key: service-ca.crt
Jul 27 17:28:35.049941 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.049909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" event={"ID":"f4417e95-0dab-43b2-aea7-e774b713f5fe","Type":"ContainerStarted","Data":"319cc39461f1ec762d65e0b57058c7edbc11659f89b13c4026a460f091fada18"}
Jul 27 17:28:35.051010 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.050987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv" event={"ID":"91c5285c-1b1f-4512-874a-5d4fc63cb925","Type":"ContainerStarted","Data":"97315accbf45e44d5284c0f3a4c6b43a7be84f738418d29802966781e9d7c225"}
Jul 27 17:28:35.052040 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.052012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc" event={"ID":"9f754619-6367-4fd6-ab8a-4c9367f5bb6f","Type":"ContainerStarted","Data":"960d2f604317ae2ce97d306d191b0e998f3494971daf78a1e6eb0a65a7dc57f5"}
Jul 27 17:28:35.053030 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.053011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w" event={"ID":"9850ae52-17e3-42bb-a0c7-c73f28f71e8d","Type":"ContainerStarted","Data":"779bd633c3b905ccfbba7ff797da9db35104730fb20e9e861af489db38c40484"}
Jul 27 17:28:35.054021 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.054003    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jwmmm" event={"ID":"d696cb3d-caf7-4bfe-876d-eca689f294f0","Type":"ContainerStarted","Data":"c9ca005173eb7e56767e315e3bbe4ad7749477f980fc399274a5b2fd85ed6c9f"}
Jul 27 17:28:35.846373 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.845727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:35.846373 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.845781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:35.846373 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.845953    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:35.846373 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.846014    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls podName:80a87ec3-31ec-4ddb-ae24-b9ef51e9c227 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:37.845995853 +0000 UTC m=+75.556731501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-xbnbm" (UID: "80a87ec3-31ec-4ddb-ae24-b9ef51e9c227") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:35.846373 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.846248    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:28:35.846373 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.846325    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls podName:ed0673a0-b03c-4ac0-8c34-08eb4a491e2a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:37.846306564 +0000 UTC m=+75.557042223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls") pod "cluster-samples-operator-6748489656-2qggm" (UID: "ed0673a0-b03c-4ac0-8c34-08eb4a491e2a") : secret "samples-operator-tls" not found
Jul 27 17:28:35.946880 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.946851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:35.946992 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:35.946980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:35.947060 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.947037    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:28:35.947126 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.947112    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:37.947093165 +0000 UTC m=+75.657828814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : configmap references non-existent config key: service-ca.crt
Jul 27 17:28:35.947176 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:35.947137    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:37.947126939 +0000 UTC m=+75.657862589 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : secret "router-metrics-certs-default" not found
Jul 27 17:28:36.059105 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:36.059060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc" event={"ID":"9f754619-6367-4fd6-ab8a-4c9367f5bb6f","Type":"ContainerStarted","Data":"9cb84500f50f7155e7e86f2658859b8b41a30b47f57712a231268b78c33471b6"}
Jul 27 17:28:36.074870 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:36.074814    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-lflhc" podStartSLOduration=0.792059759 podStartE2EDuration="2.07479362s" totalImagesPullingTime="1.282733861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:36.072813996 +0000 UTC m=+73.783549666" watchObservedRunningTime="2026-07-27 17:28:36.07479362 +0000 UTC m=+73.785529288"
Jul 27 17:28:37.862518 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:37.862483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:37.863051 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:37.862635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:37.863051 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.862754    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:37.863051 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.862814    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls podName:80a87ec3-31ec-4ddb-ae24-b9ef51e9c227 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:41.862799048 +0000 UTC m=+79.573534697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-xbnbm" (UID: "80a87ec3-31ec-4ddb-ae24-b9ef51e9c227") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:37.863051 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.862754    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:28:37.863051 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.862882    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls podName:ed0673a0-b03c-4ac0-8c34-08eb4a491e2a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:41.862869541 +0000 UTC m=+79.573605191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls") pod "cluster-samples-operator-6748489656-2qggm" (UID: "ed0673a0-b03c-4ac0-8c34-08eb4a491e2a") : secret "samples-operator-tls" not found
Jul 27 17:28:37.963646 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:37.963261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:37.963646 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:37.963406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:37.963646 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.963526    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:28:37.963646 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.963565    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:41.963547765 +0000 UTC m=+79.674283413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : configmap references non-existent config key: service-ca.crt
Jul 27 17:28:37.963646 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:37.963586    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:41.963575856 +0000 UTC m=+79.674311503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : secret "router-metrics-certs-default" not found
Jul 27 17:28:38.067451 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:38.067416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv" event={"ID":"91c5285c-1b1f-4512-874a-5d4fc63cb925","Type":"ContainerStarted","Data":"444420b4d24dd919b4b6bde722e9e085e79bcbfb44cee4cac84fb1c5c580f4dc"}
Jul 27 17:28:38.069850 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:38.069823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w" event={"ID":"9850ae52-17e3-42bb-a0c7-c73f28f71e8d","Type":"ContainerStarted","Data":"28ac57e1624e8b3c34908eab803d1adde211de8b4fa8706d8720d8bd96b8d48c"}
Jul 27 17:28:38.071911 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:38.071887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jwmmm" event={"ID":"d696cb3d-caf7-4bfe-876d-eca689f294f0","Type":"ContainerStarted","Data":"5cc4216b1a6bc4229ca236a1b173ea77557215ab37465fbeb4be2348c904cc4d"}
Jul 27 17:28:38.104121 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:38.104075    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv" podStartSLOduration=1.077007099 podStartE2EDuration="4.104027756s" totalImagesPullingTime="3.027020657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:38.10325085 +0000 UTC m=+75.813986517" watchObservedRunningTime="2026-07-27 17:28:38.104027756 +0000 UTC m=+75.814763424"
Jul 27 17:28:38.179107 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:38.179051    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w" podStartSLOduration=0.988314019 podStartE2EDuration="4.179030081s" totalImagesPullingTime="3.190716062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:38.148543666 +0000 UTC m=+75.859279335" watchObservedRunningTime="2026-07-27 17:28:38.179030081 +0000 UTC m=+75.889765750"
Jul 27 17:28:38.179362 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:38.179258    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-jwmmm" podStartSLOduration=1.135020995 podStartE2EDuration="4.17924844s" totalImagesPullingTime="3.044227445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:38.17758618 +0000 UTC m=+75.888321850" watchObservedRunningTime="2026-07-27 17:28:38.17924844 +0000 UTC m=+75.889984108"
Jul 27 17:28:39.616825 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.616788    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qdqnz"]
Jul 27 17:28:39.619962 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.619943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.626918 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.626897    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vzxqf\""
Jul 27 17:28:39.627182 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.627164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 17:28:39.627448 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.627431    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 17:28:39.642295 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.642262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qdqnz"]
Jul 27 17:28:39.679626 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.679599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.679767 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.679747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.679809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.679786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a49911cf-a59c-449e-8fc5-14128db1c9e4-crio-socket\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.679809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.679801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49911cf-a59c-449e-8fc5-14128db1c9e4-data-volume\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.679881 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.679854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9rrj\" (UniqueName: \"kubernetes.io/projected/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-api-access-b9rrj\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780195 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780387 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a49911cf-a59c-449e-8fc5-14128db1c9e4-crio-socket\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780387 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49911cf-a59c-449e-8fc5-14128db1c9e4-data-volume\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780387 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9rrj\" (UniqueName: \"kubernetes.io/projected/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-api-access-b9rrj\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780387 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780387 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a49911cf-a59c-449e-8fc5-14128db1c9e4-crio-socket\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.780559 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:39.780394    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:28:39.780559 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:39.780470    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:40.280448795 +0000 UTC m=+77.991184441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : configmap "kube-rbac-proxy" not found
Jul 27 17:28:39.780559 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:39.780470    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:39.780559 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:39.780505    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:40.280496946 +0000 UTC m=+77.991232597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:39.780559 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.780542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49911cf-a59c-449e-8fc5-14128db1c9e4-data-volume\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:39.789231 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:39.789211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9rrj\" (UniqueName: \"kubernetes.io/projected/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-api-access-b9rrj\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:40.085638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:40.085607    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/0.log"
Jul 27 17:28:40.085797 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:40.085644    2563 generic.go:356] "Generic (PLEG): container finished" podID="f4417e95-0dab-43b2-aea7-e774b713f5fe" containerID="ddf9cb073788672dde6877f7f18266b6d497a8cff8bd465e39294cc093ee04d1" exitCode=255
Jul 27 17:28:40.085797 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:40.085712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" event={"ID":"f4417e95-0dab-43b2-aea7-e774b713f5fe","Type":"ContainerDied","Data":"ddf9cb073788672dde6877f7f18266b6d497a8cff8bd465e39294cc093ee04d1"}
Jul 27 17:28:40.085930 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:40.085917    2563 scope.go:117] "RemoveContainer" containerID="ddf9cb073788672dde6877f7f18266b6d497a8cff8bd465e39294cc093ee04d1"
Jul 27 17:28:40.285228 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:40.285190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:40.285398 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:40.285247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:40.285398 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:40.285313    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:28:40.285398 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:40.285365    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:40.285522 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:40.285405    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:41.285387814 +0000 UTC m=+78.996123459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : configmap "kube-rbac-proxy" not found
Jul 27 17:28:40.285522 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:40.285420    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:41.285414512 +0000 UTC m=+78.996150157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:41.090898 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.090865    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:28:41.091325 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.091292    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/0.log"
Jul 27 17:28:41.091392 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.091324    2563 generic.go:356] "Generic (PLEG): container finished" podID="f4417e95-0dab-43b2-aea7-e774b713f5fe" containerID="2ddf77fe908fb4754a2ddddde8c10d121b7eb988da19c93ddee39fac8a1f654a" exitCode=255
Jul 27 17:28:41.091392 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.091371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" event={"ID":"f4417e95-0dab-43b2-aea7-e774b713f5fe","Type":"ContainerDied","Data":"2ddf77fe908fb4754a2ddddde8c10d121b7eb988da19c93ddee39fac8a1f654a"}
Jul 27 17:28:41.092101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.092088    2563 scope.go:117] "RemoveContainer" containerID="ddf9cb073788672dde6877f7f18266b6d497a8cff8bd465e39294cc093ee04d1"
Jul 27 17:28:41.092171 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.092154    2563 scope.go:117] "RemoveContainer" containerID="2ddf77fe908fb4754a2ddddde8c10d121b7eb988da19c93ddee39fac8a1f654a"
Jul 27 17:28:41.092386 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.092364    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fxvl2_openshift-console-operator(f4417e95-0dab-43b2-aea7-e774b713f5fe)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" podUID="f4417e95-0dab-43b2-aea7-e774b713f5fe"
Jul 27 17:28:41.294640 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.294600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:41.294835 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.294743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:41.294835 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.294748    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:41.294835 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.294809    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:43.294792202 +0000 UTC m=+81.005527847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:41.294835 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.294827    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:28:41.294992 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.294877    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:43.294860761 +0000 UTC m=+81.005596408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : configmap "kube-rbac-proxy" not found
Jul 27 17:28:41.900396 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.900360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:41.900396 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:41.900398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:41.900566 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.900502    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:41.900566 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.900554    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls podName:80a87ec3-31ec-4ddb-ae24-b9ef51e9c227 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:49.900540593 +0000 UTC m=+87.611276238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-xbnbm" (UID: "80a87ec3-31ec-4ddb-ae24-b9ef51e9c227") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:41.900665 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.900500    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:28:41.900665 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:41.900636    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls podName:ed0673a0-b03c-4ac0-8c34-08eb4a491e2a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:49.900623171 +0000 UTC m=+87.611358817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls") pod "cluster-samples-operator-6748489656-2qggm" (UID: "ed0673a0-b03c-4ac0-8c34-08eb4a491e2a") : secret "samples-operator-tls" not found
Jul 27 17:28:42.001355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:42.001300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:42.001551 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:42.001413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:42.001551 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:42.001471    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:28:42.001551 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:42.001516    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:50.0014982 +0000 UTC m=+87.712233846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : configmap references non-existent config key: service-ca.crt
Jul 27 17:28:42.001551 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:42.001536    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:50.001529588 +0000 UTC m=+87.712265232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : secret "router-metrics-certs-default" not found
Jul 27 17:28:42.096321 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:42.096294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:28:42.096708 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:42.096660    2563 scope.go:117] "RemoveContainer" containerID="2ddf77fe908fb4754a2ddddde8c10d121b7eb988da19c93ddee39fac8a1f654a"
Jul 27 17:28:42.096845 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:42.096826    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fxvl2_openshift-console-operator(f4417e95-0dab-43b2-aea7-e774b713f5fe)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" podUID="f4417e95-0dab-43b2-aea7-e774b713f5fe"
Jul 27 17:28:42.159004 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:42.158927    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjcvx_78a9220d-c136-489e-84db-36da1c4423dc/dns-node-resolver/0.log"
Jul 27 17:28:42.960167 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:42.960141    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wk5wk_56a1b52a-257c-48a7-8a73-e59e27e1a1d3/node-ca/0.log"
Jul 27 17:28:43.311853 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:43.311821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:43.312290 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:43.311897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:43.312290 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:43.311958    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:28:43.312290 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:43.311998    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:43.312290 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:43.312028    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:47.312011974 +0000 UTC m=+85.022747619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : configmap "kube-rbac-proxy" not found
Jul 27 17:28:43.312290 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:43.312043    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:47.312037686 +0000 UTC m=+85.022773331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:43.566963 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:43.566881    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dkhl7_6d90298b-c573-4d45-accc-086b982b6d8e/serve-healthcheck-canary/0.log"
Jul 27 17:28:44.359692 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:44.359656    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv_91c5285c-1b1f-4512-874a-5d4fc63cb925/kube-storage-version-migrator-operator/0.log"
Jul 27 17:28:44.587606 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:44.587572    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:44.587764 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:44.587648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:28:44.587960 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:44.587948    2563 scope.go:117] "RemoveContainer" containerID="2ddf77fe908fb4754a2ddddde8c10d121b7eb988da19c93ddee39fac8a1f654a"
Jul 27 17:28:44.588118 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:44.588099    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fxvl2_openshift-console-operator(f4417e95-0dab-43b2-aea7-e774b713f5fe)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" podUID="f4417e95-0dab-43b2-aea7-e774b713f5fe"
Jul 27 17:28:47.347048 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:47.346920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:47.347048 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:47.346995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:47.347492 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:47.347051    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:28:47.347492 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:47.347116    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:55.347100061 +0000 UTC m=+93.057835705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : configmap "kube-rbac-proxy" not found
Jul 27 17:28:47.347492 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:47.347160    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:47.347492 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:47.347221    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls podName:a49911cf-a59c-449e-8fc5-14128db1c9e4 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:55.347202788 +0000 UTC m=+93.057938445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qdqnz" (UID: "a49911cf-a59c-449e-8fc5-14128db1c9e4") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:28:49.972943 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:49.972911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:28:49.973312 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:49.973032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:49.973312 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:49.973050    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:49.973312 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:49.973121    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls podName:80a87ec3-31ec-4ddb-ae24-b9ef51e9c227 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:05.973100698 +0000 UTC m=+103.683836356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-xbnbm" (UID: "80a87ec3-31ec-4ddb-ae24-b9ef51e9c227") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:28:49.975036 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:49.975016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed0673a0-b03c-4ac0-8c34-08eb4a491e2a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2qggm\" (UID: \"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:50.073780 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:50.073754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:50.073921 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:50.073818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:50.073963 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:28:50.073943    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle podName:0e810fcc-7e49-4772-9ade-c9731493e4f7 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:06.073924271 +0000 UTC m=+103.784659917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle") pod "router-default-6dbb545db7-fnmlq" (UID: "0e810fcc-7e49-4772-9ade-c9731493e4f7") : configmap references non-existent config key: service-ca.crt
Jul 27 17:28:50.075826 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:50.075803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e810fcc-7e49-4772-9ade-c9731493e4f7-metrics-certs\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:28:50.112154 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:50.112135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"
Jul 27 17:28:50.242437 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:50.242304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm"]
Jul 27 17:28:51.127267 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:51.127229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm" event={"ID":"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a","Type":"ContainerStarted","Data":"7661ecb636e1aaedb1153440c1c3d9eff82927e0f958e6c5998cd6bd9a3f5bf7"}
Jul 27 17:28:52.134382 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:52.134328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm" event={"ID":"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a","Type":"ContainerStarted","Data":"601328ec1ddc53a59c362aae1484523b89f6218175a82e7d2cb4fa4bccf7d04e"}
Jul 27 17:28:52.134788 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:52.134398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm" event={"ID":"ed0673a0-b03c-4ac0-8c34-08eb4a491e2a","Type":"ContainerStarted","Data":"a8e9d0dd68ad08af1781f121d3abb2cb360db14ff3331a1c0fd32cd8d7048105"}
Jul 27 17:28:52.150330 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:52.150280    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2qggm" podStartSLOduration=16.691916067 podStartE2EDuration="18.150266524s" totalImagesPullingTime="1.458350457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:52.149331279 +0000 UTC m=+89.860066945" watchObservedRunningTime="2026-07-27 17:28:52.150266524 +0000 UTC m=+89.861002190"
Jul 27 17:28:55.417400 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:55.417355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:55.417779 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:55.417462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:55.418001 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:55.417970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49911cf-a59c-449e-8fc5-14128db1c9e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:55.419641 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:55.419617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49911cf-a59c-449e-8fc5-14128db1c9e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdqnz\" (UID: \"a49911cf-a59c-449e-8fc5-14128db1c9e4\") " pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:55.528892 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:55.528853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qdqnz"
Jul 27 17:28:55.654203 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:55.654174    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qdqnz"]
Jul 27 17:28:55.656997 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:28:55.656959    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49911cf_a59c_449e_8fc5_14128db1c9e4.slice/crio-da576dda89beb2172e8c8faece4f96d6ca1ad723f31fc06cfae3d987d1ec677e WatchSource:0}: Error finding container da576dda89beb2172e8c8faece4f96d6ca1ad723f31fc06cfae3d987d1ec677e: Status 404 returned error can't find the container with id da576dda89beb2172e8c8faece4f96d6ca1ad723f31fc06cfae3d987d1ec677e
Jul 27 17:28:56.148505 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:56.148423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdqnz" event={"ID":"a49911cf-a59c-449e-8fc5-14128db1c9e4","Type":"ContainerStarted","Data":"da576dda89beb2172e8c8faece4f96d6ca1ad723f31fc06cfae3d987d1ec677e"}
Jul 27 17:28:57.791843 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:28:57.791818    2563 scope.go:117] "RemoveContainer" containerID="2ddf77fe908fb4754a2ddddde8c10d121b7eb988da19c93ddee39fac8a1f654a"
Jul 27 17:29:01.167977 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.167912    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:29:01.168464 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.168024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" event={"ID":"f4417e95-0dab-43b2-aea7-e774b713f5fe","Type":"ContainerStarted","Data":"7c67e37dbb8f51f1a0796bb68e3c5fd9a414b6f5ac4212e6d63ab91cb41f86f4"}
Jul 27 17:29:01.168690 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.168633    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:29:01.170820 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.170730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdqnz" event={"ID":"a49911cf-a59c-449e-8fc5-14128db1c9e4","Type":"ContainerStarted","Data":"a6ff58fc532a8d27f15d436eafc54841ac43200913323972ff11657828915094"}
Jul 27 17:29:01.174971 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.174912    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2"
Jul 27 17:29:01.186597 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.186546    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-fxvl2" podStartSLOduration=22.799325138 podStartE2EDuration="27.186532379s" totalImagesPullingTime="4.387207241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:01.185425763 +0000 UTC m=+98.896161432" watchObservedRunningTime="2026-07-27 17:29:01.186532379 +0000 UTC m=+98.897268045"
Jul 27 17:29:01.569792 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.569664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:01.569792 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.569719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:29:01.572700 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.572672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25394289-0555-486d-a7a4-bf37a57caf8a-metrics-tls\") pod \"dns-default-sk7vl\" (UID: \"25394289-0555-486d-a7a4-bf37a57caf8a\") " pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:29:01.572921 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.572901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"image-registry-88f4bc747-qlckf\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") " pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:01.837095 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.837013    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lcj8p\""
Jul 27 17:29:01.845623 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.845595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:01.851509 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.851485    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x9hw5\""
Jul 27 17:29:01.860309 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:01.860239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:29:02.001425 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.001385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-88f4bc747-qlckf"]
Jul 27 17:29:02.005916 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:02.005885    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e620ae_b8f6_4ccd_b01c_19b402549622.slice/crio-1da0c52040ad225feede1e3cbe3c39bac8368ecaac20b518801e99fdc907a30a WatchSource:0}: Error finding container 1da0c52040ad225feede1e3cbe3c39bac8368ecaac20b518801e99fdc907a30a: Status 404 returned error can't find the container with id 1da0c52040ad225feede1e3cbe3c39bac8368ecaac20b518801e99fdc907a30a
Jul 27 17:29:02.016684 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.016661    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sk7vl"]
Jul 27 17:29:02.019707 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:02.019679    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25394289_0555_486d_a7a4_bf37a57caf8a.slice/crio-69dad24550b651037e1d69ebbdd64e9d564b87bd0a106d8da8332a7d3b075031 WatchSource:0}: Error finding container 69dad24550b651037e1d69ebbdd64e9d564b87bd0a106d8da8332a7d3b075031: Status 404 returned error can't find the container with id 69dad24550b651037e1d69ebbdd64e9d564b87bd0a106d8da8332a7d3b075031
Jul 27 17:29:02.176468 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.176350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sk7vl" event={"ID":"25394289-0555-486d-a7a4-bf37a57caf8a","Type":"ContainerStarted","Data":"69dad24550b651037e1d69ebbdd64e9d564b87bd0a106d8da8332a7d3b075031"}
Jul 27 17:29:02.178731 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.178693    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88f4bc747-qlckf" event={"ID":"90e620ae-b8f6-4ccd-b01c-19b402549622","Type":"ContainerStarted","Data":"2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188"}
Jul 27 17:29:02.178861 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.178753    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88f4bc747-qlckf" event={"ID":"90e620ae-b8f6-4ccd-b01c-19b402549622","Type":"ContainerStarted","Data":"1da0c52040ad225feede1e3cbe3c39bac8368ecaac20b518801e99fdc907a30a"}
Jul 27 17:29:02.179033 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.179012    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:02.202556 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:02.202515    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-88f4bc747-qlckf" podStartSLOduration=77.202501181 podStartE2EDuration="1m17.202501181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:02.201126611 +0000 UTC m=+99.911862278" watchObservedRunningTime="2026-07-27 17:29:02.202501181 +0000 UTC m=+99.913236863"
Jul 27 17:29:04.553175 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.553142    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"]
Jul 27 17:29:04.579359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.579304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"]
Jul 27 17:29:04.579524 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.579440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"
Jul 27 17:29:04.583386 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.583363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wzbpv\""
Jul 27 17:29:04.696816 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.696780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b94j\" (UniqueName: \"kubernetes.io/projected/5a6f3ad6-72d9-473c-b421-1a4023ef8e9a-kube-api-access-2b94j\") pod \"network-check-source-7d88cb55d8-bg78h\" (UID: \"5a6f3ad6-72d9-473c-b421-1a4023ef8e9a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"
Jul 27 17:29:04.797321 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.797286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b94j\" (UniqueName: \"kubernetes.io/projected/5a6f3ad6-72d9-473c-b421-1a4023ef8e9a-kube-api-access-2b94j\") pod \"network-check-source-7d88cb55d8-bg78h\" (UID: \"5a6f3ad6-72d9-473c-b421-1a4023ef8e9a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"
Jul 27 17:29:04.823261 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.823199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b94j\" (UniqueName: \"kubernetes.io/projected/5a6f3ad6-72d9-473c-b421-1a4023ef8e9a-kube-api-access-2b94j\") pod \"network-check-source-7d88cb55d8-bg78h\" (UID: \"5a6f3ad6-72d9-473c-b421-1a4023ef8e9a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"
Jul 27 17:29:04.891708 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:04.891679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"
Jul 27 17:29:05.056869 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:05.056836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dxf4t"
Jul 27 17:29:05.208298 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:05.208207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h"]
Jul 27 17:29:05.211210 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:05.211174    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6f3ad6_72d9_473c_b421_1a4023ef8e9a.slice/crio-4a7fbf2bb55c1a784a3fdabc63a42a499e8baf50f983b2277ea1930c2b13b368 WatchSource:0}: Error finding container 4a7fbf2bb55c1a784a3fdabc63a42a499e8baf50f983b2277ea1930c2b13b368: Status 404 returned error can't find the container with id 4a7fbf2bb55c1a784a3fdabc63a42a499e8baf50f983b2277ea1930c2b13b368
Jul 27 17:29:06.007031 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.006993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:29:06.010574 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.010548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a87ec3-31ec-4ddb-ae24-b9ef51e9c227-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xbnbm\" (UID: \"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:29:06.108143 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.108014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:06.108799 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.108778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e810fcc-7e49-4772-9ade-c9731493e4f7-service-ca-bundle\") pod \"router-default-6dbb545db7-fnmlq\" (UID: \"0e810fcc-7e49-4772-9ade-c9731493e4f7\") " pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:06.197353 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.197301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdqnz" event={"ID":"a49911cf-a59c-449e-8fc5-14128db1c9e4","Type":"ContainerStarted","Data":"c11aa69aa743f83655cf188bb6a1796e30408f25f7caa4f0b009f38ae06d5108"}
Jul 27 17:29:06.199484 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.199455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h" event={"ID":"5a6f3ad6-72d9-473c-b421-1a4023ef8e9a","Type":"ContainerStarted","Data":"da982d0f7431c7ae97116b5b9f0adf14f26d4527abd6ee452ad21acea6a7ca32"}
Jul 27 17:29:06.199618 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.199502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h" event={"ID":"5a6f3ad6-72d9-473c-b421-1a4023ef8e9a","Type":"ContainerStarted","Data":"4a7fbf2bb55c1a784a3fdabc63a42a499e8baf50f983b2277ea1930c2b13b368"}
Jul 27 17:29:06.215774 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.215721    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bg78h" podStartSLOduration=2.215703811 podStartE2EDuration="2.215703811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:06.213724588 +0000 UTC m=+103.924460256" watchObservedRunningTime="2026-07-27 17:29:06.215703811 +0000 UTC m=+103.926439483"
Jul 27 17:29:06.292975 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.292950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"
Jul 27 17:29:06.402758 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.402677    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:06.457874 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.457791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm"]
Jul 27 17:29:06.569287 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:06.569211    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dbb545db7-fnmlq"]
Jul 27 17:29:06.632293 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:06.632261    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a87ec3_31ec_4ddb_ae24_b9ef51e9c227.slice/crio-dc8b6ad2620e720e55e8d586bc4de6bd86edc7610094d9f3e5212106e5e6d052 WatchSource:0}: Error finding container dc8b6ad2620e720e55e8d586bc4de6bd86edc7610094d9f3e5212106e5e6d052: Status 404 returned error can't find the container with id dc8b6ad2620e720e55e8d586bc4de6bd86edc7610094d9f3e5212106e5e6d052
Jul 27 17:29:06.633327 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:06.633250    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e810fcc_7e49_4772_9ade_c9731493e4f7.slice/crio-0d979b3bbed5bfbb8e1f8f0a10a849aaa3a231799cd7e8456d31726b97b82df2 WatchSource:0}: Error finding container 0d979b3bbed5bfbb8e1f8f0a10a849aaa3a231799cd7e8456d31726b97b82df2: Status 404 returned error can't find the container with id 0d979b3bbed5bfbb8e1f8f0a10a849aaa3a231799cd7e8456d31726b97b82df2
Jul 27 17:29:07.205836 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.205751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm" event={"ID":"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227","Type":"ContainerStarted","Data":"dc8b6ad2620e720e55e8d586bc4de6bd86edc7610094d9f3e5212106e5e6d052"}
Jul 27 17:29:07.207750 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.207719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sk7vl" event={"ID":"25394289-0555-486d-a7a4-bf37a57caf8a","Type":"ContainerStarted","Data":"c84b2e7e6d263fffb78de34ed3b88a2afeca41731124c80a514ccd47ffeffef8"}
Jul 27 17:29:07.207878 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.207771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sk7vl" event={"ID":"25394289-0555-486d-a7a4-bf37a57caf8a","Type":"ContainerStarted","Data":"772f4f0b30a0b36d97b2e7bad265f3c609e2bb5d2bb147a0c8f11a56e88124e0"}
Jul 27 17:29:07.207878 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.207861    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:29:07.209501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.209476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dbb545db7-fnmlq" event={"ID":"0e810fcc-7e49-4772-9ade-c9731493e4f7","Type":"ContainerStarted","Data":"81d43800af6fda6a258bf096def21ef8018b13dfc9005f3f49edd3fbeafdcf41"}
Jul 27 17:29:07.209614 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.209522    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dbb545db7-fnmlq" event={"ID":"0e810fcc-7e49-4772-9ade-c9731493e4f7","Type":"ContainerStarted","Data":"0d979b3bbed5bfbb8e1f8f0a10a849aaa3a231799cd7e8456d31726b97b82df2"}
Jul 27 17:29:07.227564 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.227521    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sk7vl" podStartSLOduration=65.583527049 podStartE2EDuration="1m10.227503529s" totalImagesPullingTime="4.64397648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:07.226711703 +0000 UTC m=+104.937447372" watchObservedRunningTime="2026-07-27 17:29:07.227503529 +0000 UTC m=+104.938239196"
Jul 27 17:29:07.244095 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.244037    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6dbb545db7-fnmlq" podStartSLOduration=33.24402258 podStartE2EDuration="33.24402258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:07.243167706 +0000 UTC m=+104.953903373" watchObservedRunningTime="2026-07-27 17:29:07.24402258 +0000 UTC m=+104.954758247"
Jul 27 17:29:07.403089 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.403055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:07.406046 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:07.406020    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:08.212839 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:08.212801    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:08.214121 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:08.214097    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6dbb545db7-fnmlq"
Jul 27 17:29:11.225230 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:11.225196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdqnz" event={"ID":"a49911cf-a59c-449e-8fc5-14128db1c9e4","Type":"ContainerStarted","Data":"14a38053d3039b8f46aea84af88d5f9822d5acf627591ccc3d6baec1f8537174"}
Jul 27 17:29:11.226586 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:11.226566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm" event={"ID":"80a87ec3-31ec-4ddb-ae24-b9ef51e9c227","Type":"ContainerStarted","Data":"033a03d07a55663fc6878d90e4c1801979e9ddc42789383b2881f625f5b60a71"}
Jul 27 17:29:11.243882 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:11.243834    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qdqnz" podStartSLOduration=17.942317648 podStartE2EDuration="32.243820458s" totalImagesPullingTime="14.30150281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:11.243331278 +0000 UTC m=+108.954066948" watchObservedRunningTime="2026-07-27 17:29:11.243820458 +0000 UTC m=+108.954556122"
Jul 27 17:29:11.259089 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:11.259047    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xbnbm" podStartSLOduration=33.721956422 podStartE2EDuration="37.259038306s" totalImagesPullingTime="3.537081884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:11.258237519 +0000 UTC m=+108.968973186" watchObservedRunningTime="2026-07-27 17:29:11.259038306 +0000 UTC m=+108.969773973"
Jul 27 17:29:17.214780 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:17.214748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sk7vl"
Jul 27 17:29:23.167467 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.167436    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f4k9p"]
Jul 27 17:29:23.170696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.170680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.173981 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.173957    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fhlh7\""
Jul 27 17:29:23.173981 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.173971    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 17:29:23.174140 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.174019    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 17:29:23.174140 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.174081    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 17:29:23.174217 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.174148    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 17:29:23.186707 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.186689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:23.241827 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.241799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.241960 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.241841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-tls\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.241960 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.241871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-metrics-client-ca\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.242080 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.242052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-sys\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.242171 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.242151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-wtmp\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.242244 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.242223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-textfile\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.242299 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.242272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-root\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.242378 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.242322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.242458 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.242441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2cd\" (UniqueName: \"kubernetes.io/projected/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-kube-api-access-nc2cd\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343155 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-sys\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-wtmp\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-sys\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-textfile\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-wtmp\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-root\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-root\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2cd\" (UniqueName: \"kubernetes.io/projected/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-kube-api-access-nc2cd\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343780 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-tls\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343780 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-metrics-client-ca\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343780 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:29:23.343628    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 17:29:23.343780 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.343634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-textfile\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.343780 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:29:23.343698    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-tls podName:ad14d593-83c0-4c43-8837-9aaf1fb10e8e nodeName:}" failed. No retries permitted until 2026-07-27 17:29:23.843679424 +0000 UTC m=+121.554415081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-tls") pod "node-exporter-f4k9p" (UID: "ad14d593-83c0-4c43-8837-9aaf1fb10e8e") : secret "node-exporter-tls" not found
Jul 27 17:29:23.344048 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.344025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.344104 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.344084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-metrics-client-ca\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.345698 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.345679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.353615 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.353596    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2cd\" (UniqueName: \"kubernetes.io/projected/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-kube-api-access-nc2cd\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.847011 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.846976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-tls\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:23.849181 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:23.849156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ad14d593-83c0-4c43-8837-9aaf1fb10e8e-node-exporter-tls\") pod \"node-exporter-f4k9p\" (UID: \"ad14d593-83c0-4c43-8837-9aaf1fb10e8e\") " pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:24.079640 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:24.079609    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f4k9p"
Jul 27 17:29:24.090999 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:24.090972    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad14d593_83c0_4c43_8837_9aaf1fb10e8e.slice/crio-995d62ef5381554fb824fb7668908799336bb155b1393b2baf7c20d4bb56f596 WatchSource:0}: Error finding container 995d62ef5381554fb824fb7668908799336bb155b1393b2baf7c20d4bb56f596: Status 404 returned error can't find the container with id 995d62ef5381554fb824fb7668908799336bb155b1393b2baf7c20d4bb56f596
Jul 27 17:29:24.274800 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:24.274717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4k9p" event={"ID":"ad14d593-83c0-4c43-8837-9aaf1fb10e8e","Type":"ContainerStarted","Data":"995d62ef5381554fb824fb7668908799336bb155b1393b2baf7c20d4bb56f596"}
Jul 27 17:29:25.279212 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:25.279178    2563 generic.go:356] "Generic (PLEG): container finished" podID="ad14d593-83c0-4c43-8837-9aaf1fb10e8e" containerID="753f1a73fad2a5d7107f8207e58c3531738421d8d6e4f46b5bb4d81ff788b8ef" exitCode=0
Jul 27 17:29:25.279581 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:25.279220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4k9p" event={"ID":"ad14d593-83c0-4c43-8837-9aaf1fb10e8e","Type":"ContainerDied","Data":"753f1a73fad2a5d7107f8207e58c3531738421d8d6e4f46b5bb4d81ff788b8ef"}
Jul 27 17:29:26.285113 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:26.285080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4k9p" event={"ID":"ad14d593-83c0-4c43-8837-9aaf1fb10e8e","Type":"ContainerStarted","Data":"9a9bc1080885efd84f94dde194220a2624cb4aa846c0a9730708f8ae7bcc6e5f"}
Jul 27 17:29:26.285492 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:26.285135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4k9p" event={"ID":"ad14d593-83c0-4c43-8837-9aaf1fb10e8e","Type":"ContainerStarted","Data":"f6bcd600615974bffd48ce41024786920946903837124f6040bad2dd79c32aee"}
Jul 27 17:29:26.308438 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:26.308394    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f4k9p" podStartSLOduration=2.244013417 podStartE2EDuration="3.308380064s" totalImagesPullingTime="1.002090684s" totalInitContainerRuntime="62.275963ms" isStatefulPod=true podCreationTimestamp="2026-07-27 17:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:26.307460898 +0000 UTC m=+124.018196566" watchObservedRunningTime="2026-07-27 17:29:26.308380064 +0000 UTC m=+124.019115713"
Jul 27 17:29:27.478102 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.478068    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-88f4bc747-qlckf"]
Jul 27 17:29:27.509166 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.509140    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6785ff5849-q65rt"]
Jul 27 17:29:27.513676 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.513659    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.517932 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.517913    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 17:29:27.517932 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.517916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 17:29:27.519424 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.519402    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 17:29:27.519537 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.519473    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f45gz\""
Jul 27 17:29:27.519537 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.519489    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5kmaiai2mi6ao\""
Jul 27 17:29:27.519537 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.519529    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 17:29:27.539789 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.539767    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6785ff5849-q65rt"]
Jul 27 17:29:27.572907 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.572883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f2d95b69-a34f-4a3d-80e7-d7d84527813b-metrics-server-audit-profiles\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.573002 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.572917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kqhz\" (UniqueName: \"kubernetes.io/projected/f2d95b69-a34f-4a3d-80e7-d7d84527813b-kube-api-access-9kqhz\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.573002 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.572961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-secret-metrics-server-tls\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.573078 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.573008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-secret-metrics-server-client-certs\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.573078 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.573062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2d95b69-a34f-4a3d-80e7-d7d84527813b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.573143 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.573094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f2d95b69-a34f-4a3d-80e7-d7d84527813b-audit-log\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.573143 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.573120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-client-ca-bundle\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.673735 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.673708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-secret-metrics-server-client-certs\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.673842 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.673743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2d95b69-a34f-4a3d-80e7-d7d84527813b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.673842 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.673762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f2d95b69-a34f-4a3d-80e7-d7d84527813b-audit-log\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.673842 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.673807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-client-ca-bundle\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.673997 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.673843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f2d95b69-a34f-4a3d-80e7-d7d84527813b-metrics-server-audit-profiles\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.673997 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.673973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kqhz\" (UniqueName: \"kubernetes.io/projected/f2d95b69-a34f-4a3d-80e7-d7d84527813b-kube-api-access-9kqhz\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.674103 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.674027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-secret-metrics-server-tls\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.674161 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.674113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f2d95b69-a34f-4a3d-80e7-d7d84527813b-audit-log\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.674559 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.674534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2d95b69-a34f-4a3d-80e7-d7d84527813b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.674818 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.674799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f2d95b69-a34f-4a3d-80e7-d7d84527813b-metrics-server-audit-profiles\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.676520 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.676497    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-client-ca-bundle\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.676647 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.676629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-secret-metrics-server-client-certs\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.676735 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.676715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f2d95b69-a34f-4a3d-80e7-d7d84527813b-secret-metrics-server-tls\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.686139 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.686120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kqhz\" (UniqueName: \"kubernetes.io/projected/f2d95b69-a34f-4a3d-80e7-d7d84527813b-kube-api-access-9kqhz\") pod \"metrics-server-6785ff5849-q65rt\" (UID: \"f2d95b69-a34f-4a3d-80e7-d7d84527813b\") " pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.822871 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.822846    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:27.962912 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:27.962880    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6785ff5849-q65rt"]
Jul 27 17:29:27.965927 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:27.965906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d95b69_a34f_4a3d_80e7_d7d84527813b.slice/crio-197e4d3c4a6e7ab4ff097efd71e88059f88948aecb9022c7590c3c529f07c330 WatchSource:0}: Error finding container 197e4d3c4a6e7ab4ff097efd71e88059f88948aecb9022c7590c3c529f07c330: Status 404 returned error can't find the container with id 197e4d3c4a6e7ab4ff097efd71e88059f88948aecb9022c7590c3c529f07c330
Jul 27 17:29:28.292798 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:28.292765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt" event={"ID":"f2d95b69-a34f-4a3d-80e7-d7d84527813b","Type":"ContainerStarted","Data":"197e4d3c4a6e7ab4ff097efd71e88059f88948aecb9022c7590c3c529f07c330"}
Jul 27 17:29:31.304377 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:31.304331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt" event={"ID":"f2d95b69-a34f-4a3d-80e7-d7d84527813b","Type":"ContainerStarted","Data":"2266521f08ccb191756f3c65489f0129694f7d2c52f86c981a557b3738711db7"}
Jul 27 17:29:31.321098 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:31.321046    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt" podStartSLOduration=1.5220042459999998 podStartE2EDuration="4.321030871s" totalImagesPullingTime="2.799026625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:31.319930026 +0000 UTC m=+129.030665693" watchObservedRunningTime="2026-07-27 17:29:31.321030871 +0000 UTC m=+129.031766537"
Jul 27 17:29:32.613041 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:32.613006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:29:32.615224 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:32.615202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269531c4-3fdf-4ceb-8046-2228ac3c143d-metrics-certs\") pod \"network-metrics-daemon-n44f9\" (UID: \"269531c4-3fdf-4ceb-8046-2228ac3c143d\") " pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:29:32.721618 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:32.721591    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6sz2v\""
Jul 27 17:29:32.730199 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:32.730174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n44f9"
Jul 27 17:29:32.906951 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:32.906796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n44f9"]
Jul 27 17:29:32.909594 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:29:32.909565    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269531c4_3fdf_4ceb_8046_2228ac3c143d.slice/crio-8a2156893f326b3cca98164d4e3c67dbdcfedfdaf7f52440dd7c0cc1be5f992d WatchSource:0}: Error finding container 8a2156893f326b3cca98164d4e3c67dbdcfedfdaf7f52440dd7c0cc1be5f992d: Status 404 returned error can't find the container with id 8a2156893f326b3cca98164d4e3c67dbdcfedfdaf7f52440dd7c0cc1be5f992d
Jul 27 17:29:33.312616 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:33.312584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n44f9" event={"ID":"269531c4-3fdf-4ceb-8046-2228ac3c143d","Type":"ContainerStarted","Data":"8a2156893f326b3cca98164d4e3c67dbdcfedfdaf7f52440dd7c0cc1be5f992d"}
Jul 27 17:29:35.323445 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:35.323406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n44f9" event={"ID":"269531c4-3fdf-4ceb-8046-2228ac3c143d","Type":"ContainerStarted","Data":"f56968422701f2e4cf228f477352915a8f4215882fcea069437788b83e8414f6"}
Jul 27 17:29:35.323791 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:35.323461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n44f9" event={"ID":"269531c4-3fdf-4ceb-8046-2228ac3c143d","Type":"ContainerStarted","Data":"14cfc13d48733463929abbd98450c80e571db3e8dadfa8e47640f6d64ad31a0a"}
Jul 27 17:29:35.349887 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:35.349841    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n44f9" podStartSLOduration=131.051106427 podStartE2EDuration="2m12.349827763s" totalImagesPullingTime="1.298721336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:35.349049176 +0000 UTC m=+133.059784842" watchObservedRunningTime="2026-07-27 17:29:35.349827763 +0000 UTC m=+133.060563430"
Jul 27 17:29:44.354589 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:44.354556    2563 generic.go:356] "Generic (PLEG): container finished" podID="d696cb3d-caf7-4bfe-876d-eca689f294f0" containerID="5cc4216b1a6bc4229ca236a1b173ea77557215ab37465fbeb4be2348c904cc4d" exitCode=0
Jul 27 17:29:44.354946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:44.354629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jwmmm" event={"ID":"d696cb3d-caf7-4bfe-876d-eca689f294f0","Type":"ContainerDied","Data":"5cc4216b1a6bc4229ca236a1b173ea77557215ab37465fbeb4be2348c904cc4d"}
Jul 27 17:29:44.354946 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:44.354912    2563 scope.go:117] "RemoveContainer" containerID="5cc4216b1a6bc4229ca236a1b173ea77557215ab37465fbeb4be2348c904cc4d"
Jul 27 17:29:45.360649 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:45.360614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jwmmm" event={"ID":"d696cb3d-caf7-4bfe-876d-eca689f294f0","Type":"ContainerStarted","Data":"661d9683710167e5252a1be67d2493c751f2788ef7400aee7ba76fd5ee60ca03"}
Jul 27 17:29:47.823902 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:47.823861    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:47.824317 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:47.823926    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:29:49.375718 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:49.375686    2563 generic.go:356] "Generic (PLEG): container finished" podID="91c5285c-1b1f-4512-874a-5d4fc63cb925" containerID="444420b4d24dd919b4b6bde722e9e085e79bcbfb44cee4cac84fb1c5c580f4dc" exitCode=0
Jul 27 17:29:49.376091 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:49.375757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv" event={"ID":"91c5285c-1b1f-4512-874a-5d4fc63cb925","Type":"ContainerDied","Data":"444420b4d24dd919b4b6bde722e9e085e79bcbfb44cee4cac84fb1c5c580f4dc"}
Jul 27 17:29:49.376091 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:49.376078    2563 scope.go:117] "RemoveContainer" containerID="444420b4d24dd919b4b6bde722e9e085e79bcbfb44cee4cac84fb1c5c580f4dc"
Jul 27 17:29:50.381038 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:50.381000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv" event={"ID":"91c5285c-1b1f-4512-874a-5d4fc63cb925","Type":"ContainerStarted","Data":"7c6a4fdc4bdc7c2d476bcc977580e43f69ddd9c8046887c8426c22a1dedc5043"}
Jul 27 17:29:52.500535 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.500491    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-88f4bc747-qlckf" podUID="90e620ae-b8f6-4ccd-b01c-19b402549622" containerName="registry" containerID="cri-o://2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188" gracePeriod=30
Jul 27 17:29:52.746814 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.746783    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:52.850735 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.850901 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850764    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd7xl\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-kube-api-access-zd7xl\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.850901 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-trusted-ca\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.850901 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850814    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-certificates\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.850901 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850848    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-image-registry-private-configuration\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.850901 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-installation-pull-secrets\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.851157 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90e620ae-b8f6-4ccd-b01c-19b402549622-ca-trust-extracted\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.851157 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.850946    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-bound-sa-token\") pod \"90e620ae-b8f6-4ccd-b01c-19b402549622\" (UID: \"90e620ae-b8f6-4ccd-b01c-19b402549622\") "
Jul 27 17:29:52.851367 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.851269    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:29:52.851367 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.851294    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:29:52.853168 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.853136    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:29:52.853532 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.853498    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:29:52.853642 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.853551    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-kube-api-access-zd7xl" (OuterVolumeSpecName: "kube-api-access-zd7xl") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "kube-api-access-zd7xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:29:52.853642 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.853615    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:29:52.853716 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.853649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:29:52.859587 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.859554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e620ae-b8f6-4ccd-b01c-19b402549622-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "90e620ae-b8f6-4ccd-b01c-19b402549622" (UID: "90e620ae-b8f6-4ccd-b01c-19b402549622"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:29:52.951379 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951351    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-bound-sa-token\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951379 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951377    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951390    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd7xl\" (UniqueName: \"kubernetes.io/projected/90e620ae-b8f6-4ccd-b01c-19b402549622-kube-api-access-zd7xl\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951403    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-trusted-ca\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951413    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90e620ae-b8f6-4ccd-b01c-19b402549622-registry-certificates\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951422    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-image-registry-private-configuration\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951430    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90e620ae-b8f6-4ccd-b01c-19b402549622-installation-pull-secrets\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:52.951502 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:52.951438    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90e620ae-b8f6-4ccd-b01c-19b402549622-ca-trust-extracted\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:29:53.392647 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.392617    2563 generic.go:356] "Generic (PLEG): container finished" podID="90e620ae-b8f6-4ccd-b01c-19b402549622" containerID="2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188" exitCode=0
Jul 27 17:29:53.392798 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.392659    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88f4bc747-qlckf"
Jul 27 17:29:53.392798 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.392696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88f4bc747-qlckf" event={"ID":"90e620ae-b8f6-4ccd-b01c-19b402549622","Type":"ContainerDied","Data":"2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188"}
Jul 27 17:29:53.392798 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.392733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88f4bc747-qlckf" event={"ID":"90e620ae-b8f6-4ccd-b01c-19b402549622","Type":"ContainerDied","Data":"1da0c52040ad225feede1e3cbe3c39bac8368ecaac20b518801e99fdc907a30a"}
Jul 27 17:29:53.392798 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.392752    2563 scope.go:117] "RemoveContainer" containerID="2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188"
Jul 27 17:29:53.406120 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.406094    2563 scope.go:117] "RemoveContainer" containerID="2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188"
Jul 27 17:29:53.406410 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:29:53.406387    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188\": container with ID starting with 2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188 not found: ID does not exist" containerID="2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188"
Jul 27 17:29:53.406476 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.406418    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188"} err="failed to get container status \"2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188\": rpc error: code = NotFound desc = could not find container \"2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188\": container with ID starting with 2d238178e68c798c70f6faa33b6400afe913c5138b55931450efd1fd4778f188 not found: ID does not exist"
Jul 27 17:29:53.415301 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.415280    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-88f4bc747-qlckf"]
Jul 27 17:29:53.419207 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:53.419188    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-88f4bc747-qlckf"]
Jul 27 17:29:54.795934 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:29:54.795901    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90e620ae-b8f6-4ccd-b01c-19b402549622" path="/var/lib/kubelet/pods/90e620ae-b8f6-4ccd-b01c-19b402549622/volumes"
Jul 27 17:30:03.428414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:30:03.428330    2563 generic.go:356] "Generic (PLEG): container finished" podID="9850ae52-17e3-42bb-a0c7-c73f28f71e8d" containerID="28ac57e1624e8b3c34908eab803d1adde211de8b4fa8706d8720d8bd96b8d48c" exitCode=0
Jul 27 17:30:03.428414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:30:03.428400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w" event={"ID":"9850ae52-17e3-42bb-a0c7-c73f28f71e8d","Type":"ContainerDied","Data":"28ac57e1624e8b3c34908eab803d1adde211de8b4fa8706d8720d8bd96b8d48c"}
Jul 27 17:30:03.428774 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:30:03.428686    2563 scope.go:117] "RemoveContainer" containerID="28ac57e1624e8b3c34908eab803d1adde211de8b4fa8706d8720d8bd96b8d48c"
Jul 27 17:30:04.433993 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:30:04.433959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5mx9w" event={"ID":"9850ae52-17e3-42bb-a0c7-c73f28f71e8d","Type":"ContainerStarted","Data":"8bc887e699f8b98d3cc09a20849f0475830a1fe242db50d9fbb1eed36115885c"}
Jul 27 17:30:07.828465 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:30:07.828438    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:30:07.832226 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:30:07.832204    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6785ff5849-q65rt"
Jul 27 17:32:22.691226 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:32:22.691198    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:32:22.702896 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:32:22.702873    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:32:22.703558 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:32:22.703542    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 17:33:00.829630 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.829600    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"]
Jul 27 17:33:00.831868 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.829838    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e620ae-b8f6-4ccd-b01c-19b402549622" containerName="registry"
Jul 27 17:33:00.831868 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.829849    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e620ae-b8f6-4ccd-b01c-19b402549622" containerName="registry"
Jul 27 17:33:00.831868 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.829899    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="90e620ae-b8f6-4ccd-b01c-19b402549622" containerName="registry"
Jul 27 17:33:00.832710 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.832693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:00.834516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.834496    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 17:33:00.835127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.835109    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 17:33:00.835224 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.835125    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 17:33:00.835224 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.835138    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 27 17:33:00.843122 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.843101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"]
Jul 27 17:33:00.884377 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.884330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54df5c85f6-lxv94\" (UID: \"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:00.884465 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.884391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p5sl\" (UniqueName: \"kubernetes.io/projected/87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b-kube-api-access-4p5sl\") pod \"managed-serviceaccount-addon-agent-54df5c85f6-lxv94\" (UID: \"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:00.956161 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.956137    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"]
Jul 27 17:33:00.959029 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.959012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:00.960939 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.960923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 27 17:33:00.968147 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.968126    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"]
Jul 27 17:33:00.984982 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.984963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54df5c85f6-lxv94\" (UID: \"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:00.985065 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.984999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p5sl\" (UniqueName: \"kubernetes.io/projected/87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b-kube-api-access-4p5sl\") pod \"managed-serviceaccount-addon-agent-54df5c85f6-lxv94\" (UID: \"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:00.987241 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.987215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54df5c85f6-lxv94\" (UID: \"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:00.992456 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:00.992435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p5sl\" (UniqueName: \"kubernetes.io/projected/87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b-kube-api-access-4p5sl\") pod \"managed-serviceaccount-addon-agent-54df5c85f6-lxv94\" (UID: \"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:01.086122 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.086073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97cde0a0-432e-4179-bc7e-2bc940eea55d-klusterlet-config\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.086122 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.086112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ngmx\" (UniqueName: \"kubernetes.io/projected/97cde0a0-432e-4179-bc7e-2bc940eea55d-kube-api-access-8ngmx\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.086253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.086138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97cde0a0-432e-4179-bc7e-2bc940eea55d-tmp\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.151382 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.151363    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"
Jul 27 17:33:01.187100 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.187076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97cde0a0-432e-4179-bc7e-2bc940eea55d-klusterlet-config\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.187220 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.187119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ngmx\" (UniqueName: \"kubernetes.io/projected/97cde0a0-432e-4179-bc7e-2bc940eea55d-kube-api-access-8ngmx\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.187220 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.187151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97cde0a0-432e-4179-bc7e-2bc940eea55d-tmp\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.187531 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.187508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97cde0a0-432e-4179-bc7e-2bc940eea55d-tmp\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.189501 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.189479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97cde0a0-432e-4179-bc7e-2bc940eea55d-klusterlet-config\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.195982 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.195959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ngmx\" (UniqueName: \"kubernetes.io/projected/97cde0a0-432e-4179-bc7e-2bc940eea55d-kube-api-access-8ngmx\") pod \"klusterlet-addon-workmgr-76dd77874f-9hlwh\" (UID: \"97cde0a0-432e-4179-bc7e-2bc940eea55d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.267536 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.267506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:01.283041 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.283014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94"]
Jul 27 17:33:01.286035 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:33:01.286012    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87ebbc3f_5be9_44c4_88b2_6ebd5fa9ab3b.slice/crio-10f028cd0515b87dae25c89aa88061c3dfaf9def37efecb1bec11cdc2693f0ca WatchSource:0}: Error finding container 10f028cd0515b87dae25c89aa88061c3dfaf9def37efecb1bec11cdc2693f0ca: Status 404 returned error can't find the container with id 10f028cd0515b87dae25c89aa88061c3dfaf9def37efecb1bec11cdc2693f0ca
Jul 27 17:33:01.287726 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.287708    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:33:01.389079 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:01.389046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"]
Jul 27 17:33:01.392628 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:33:01.392601    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97cde0a0_432e_4179_bc7e_2bc940eea55d.slice/crio-edd7ff39f861246ab9e0f7c34963b352ef86e77e4d9a6c74ae55a6c65402db5b WatchSource:0}: Error finding container edd7ff39f861246ab9e0f7c34963b352ef86e77e4d9a6c74ae55a6c65402db5b: Status 404 returned error can't find the container with id edd7ff39f861246ab9e0f7c34963b352ef86e77e4d9a6c74ae55a6c65402db5b
Jul 27 17:33:02.033057 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:02.033025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94" event={"ID":"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b","Type":"ContainerStarted","Data":"10f028cd0515b87dae25c89aa88061c3dfaf9def37efecb1bec11cdc2693f0ca"}
Jul 27 17:33:02.034020 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:02.033989    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh" event={"ID":"97cde0a0-432e-4179-bc7e-2bc940eea55d","Type":"ContainerStarted","Data":"edd7ff39f861246ab9e0f7c34963b352ef86e77e4d9a6c74ae55a6c65402db5b"}
Jul 27 17:33:06.053412 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:06.053373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94" event={"ID":"87ebbc3f-5be9-44c4-88b2-6ebd5fa9ab3b","Type":"ContainerStarted","Data":"d8f2741e1791b8d62d81f1d8066da36fcaf14a6a978733f96fab3bf1f093f5b6"}
Jul 27 17:33:06.054855 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:06.054823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh" event={"ID":"97cde0a0-432e-4179-bc7e-2bc940eea55d","Type":"ContainerStarted","Data":"f3543344e803538f5eee0cff3a79f0d76cb3c71bbbc1b29884b097ad3d0d54e3"}
Jul 27 17:33:06.055035 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:06.055019    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:06.056648 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:06.056624    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh"
Jul 27 17:33:06.068944 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:06.068904    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54df5c85f6-lxv94" podStartSLOduration=1.8241074689999999 podStartE2EDuration="6.068882563s" totalImagesPullingTime="4.244775094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:06.06793501 +0000 UTC m=+343.778670700" watchObservedRunningTime="2026-07-27 17:33:06.068882563 +0000 UTC m=+343.779618229"
Jul 27 17:33:06.087469 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:06.087426    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76dd77874f-9hlwh" podStartSLOduration=1.9382610850000002 podStartE2EDuration="6.087415316s" totalImagesPullingTime="4.149154231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:06.085971846 +0000 UTC m=+343.796707536" watchObservedRunningTime="2026-07-27 17:33:06.087415316 +0000 UTC m=+343.798150981"
Jul 27 17:33:54.026842 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.026809    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"]
Jul 27 17:33:54.030263 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.030240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.032292 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.032270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 27 17:33:54.032805 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.032789    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 27 17:33:54.032853 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.032810    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-npbxr\""
Jul 27 17:33:54.037515 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.037487    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"]
Jul 27 17:33:54.060000 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.059975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.060072 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.060007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.060072 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.060052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp44v\" (UniqueName: \"kubernetes.io/projected/7c37eaaf-1e10-42f1-aeb9-d7771174f508-kube-api-access-fp44v\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.160468 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.160439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.160593 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.160471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.160593 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.160583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp44v\" (UniqueName: \"kubernetes.io/projected/7c37eaaf-1e10-42f1-aeb9-d7771174f508-kube-api-access-fp44v\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.160876 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.160856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.160938 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.160853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.168363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.168330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp44v\" (UniqueName: \"kubernetes.io/projected/7c37eaaf-1e10-42f1-aeb9-d7771174f508-kube-api-access-fp44v\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.339970 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.339931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:33:54.463755 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:54.463575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"]
Jul 27 17:33:54.466227 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:33:54.466203    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c37eaaf_1e10_42f1_aeb9_d7771174f508.slice/crio-0497fb4a28c4cdc8fca73d83600dc553c260389122f5854335f804fd08d563f1 WatchSource:0}: Error finding container 0497fb4a28c4cdc8fca73d83600dc553c260389122f5854335f804fd08d563f1: Status 404 returned error can't find the container with id 0497fb4a28c4cdc8fca73d83600dc553c260389122f5854335f804fd08d563f1
Jul 27 17:33:55.223981 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:33:55.223936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8" event={"ID":"7c37eaaf-1e10-42f1-aeb9-d7771174f508","Type":"ContainerStarted","Data":"0497fb4a28c4cdc8fca73d83600dc553c260389122f5854335f804fd08d563f1"}
Jul 27 17:34:01.252196 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:01.252101    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerID="1ec06d9597dbccfb4aaabfaebc6e749c827836bd5cee7d70ac899a7bb014f290" exitCode=0
Jul 27 17:34:01.252666 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:01.252186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8" event={"ID":"7c37eaaf-1e10-42f1-aeb9-d7771174f508","Type":"ContainerDied","Data":"1ec06d9597dbccfb4aaabfaebc6e749c827836bd5cee7d70ac899a7bb014f290"}
Jul 27 17:34:04.265658 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:04.265624    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerID="07dbb1cced882e17e0703dd4499951d488aa1de751a50d95ce8e46b792b6ff3c" exitCode=0
Jul 27 17:34:04.266050 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:04.265710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8" event={"ID":"7c37eaaf-1e10-42f1-aeb9-d7771174f508","Type":"ContainerDied","Data":"07dbb1cced882e17e0703dd4499951d488aa1de751a50d95ce8e46b792b6ff3c"}
Jul 27 17:34:12.298529 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:12.298500    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerID="3b23cb86f26493d522f37283b47fc97673be70af7cbf7a2fd27e0e140c31a8ce" exitCode=0
Jul 27 17:34:12.298829 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:12.298591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8" event={"ID":"7c37eaaf-1e10-42f1-aeb9-d7771174f508","Type":"ContainerDied","Data":"3b23cb86f26493d522f37283b47fc97673be70af7cbf7a2fd27e0e140c31a8ce"}
Jul 27 17:34:13.423879 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.423849    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:34:13.507763 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.507734    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-util\") pod \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") "
Jul 27 17:34:13.507890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.507769    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-bundle\") pod \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") "
Jul 27 17:34:13.507890 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.507826    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp44v\" (UniqueName: \"kubernetes.io/projected/7c37eaaf-1e10-42f1-aeb9-d7771174f508-kube-api-access-fp44v\") pod \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\" (UID: \"7c37eaaf-1e10-42f1-aeb9-d7771174f508\") "
Jul 27 17:34:13.508555 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.508530    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-bundle" (OuterVolumeSpecName: "bundle") pod "7c37eaaf-1e10-42f1-aeb9-d7771174f508" (UID: "7c37eaaf-1e10-42f1-aeb9-d7771174f508"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:34:13.509863 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.509843    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c37eaaf-1e10-42f1-aeb9-d7771174f508-kube-api-access-fp44v" (OuterVolumeSpecName: "kube-api-access-fp44v") pod "7c37eaaf-1e10-42f1-aeb9-d7771174f508" (UID: "7c37eaaf-1e10-42f1-aeb9-d7771174f508"). InnerVolumeSpecName "kube-api-access-fp44v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:34:13.513724 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.513699    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-util" (OuterVolumeSpecName: "util") pod "7c37eaaf-1e10-42f1-aeb9-d7771174f508" (UID: "7c37eaaf-1e10-42f1-aeb9-d7771174f508"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:34:13.608363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.608297    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-util\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:34:13.608363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.608324    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c37eaaf-1e10-42f1-aeb9-d7771174f508-bundle\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:34:13.608363 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:13.608349    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp44v\" (UniqueName: \"kubernetes.io/projected/7c37eaaf-1e10-42f1-aeb9-d7771174f508-kube-api-access-fp44v\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:34:14.306416 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:14.306385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8" event={"ID":"7c37eaaf-1e10-42f1-aeb9-d7771174f508","Type":"ContainerDied","Data":"0497fb4a28c4cdc8fca73d83600dc553c260389122f5854335f804fd08d563f1"}
Jul 27 17:34:14.306548 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:14.306437    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0497fb4a28c4cdc8fca73d83600dc553c260389122f5854335f804fd08d563f1"
Jul 27 17:34:14.306548 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:14.306474    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5dfbl8"
Jul 27 17:34:16.720997 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.720961    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"]
Jul 27 17:34:16.721404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721286    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="pull"
Jul 27 17:34:16.721404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721298    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="pull"
Jul 27 17:34:16.721404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721306    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="extract"
Jul 27 17:34:16.721404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721312    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="extract"
Jul 27 17:34:16.721404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721318    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="util"
Jul 27 17:34:16.721404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721324    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="util"
Jul 27 17:34:16.721598 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.721420    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c37eaaf-1e10-42f1-aeb9-d7771174f508" containerName="extract"
Jul 27 17:34:16.768864 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.768839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"]
Jul 27 17:34:16.768972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.768925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:16.772204 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.772182    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 27 17:34:16.772204 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.772201    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 27 17:34:16.772477 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.772456    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 27 17:34:16.772773 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.772761    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-m2c24\""
Jul 27 17:34:16.830854 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.830833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d326762b-a3ee-484d-89b2-47a64a7800c7-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj\" (UID: \"d326762b-a3ee-484d-89b2-47a64a7800c7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:16.830941 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.830875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7w8z\" (UniqueName: \"kubernetes.io/projected/d326762b-a3ee-484d-89b2-47a64a7800c7-kube-api-access-l7w8z\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj\" (UID: \"d326762b-a3ee-484d-89b2-47a64a7800c7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:16.931246 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.931223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7w8z\" (UniqueName: \"kubernetes.io/projected/d326762b-a3ee-484d-89b2-47a64a7800c7-kube-api-access-l7w8z\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj\" (UID: \"d326762b-a3ee-484d-89b2-47a64a7800c7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:16.931352 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.931270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d326762b-a3ee-484d-89b2-47a64a7800c7-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj\" (UID: \"d326762b-a3ee-484d-89b2-47a64a7800c7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:16.933495 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.933468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d326762b-a3ee-484d-89b2-47a64a7800c7-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj\" (UID: \"d326762b-a3ee-484d-89b2-47a64a7800c7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:16.941384 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:16.941366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7w8z\" (UniqueName: \"kubernetes.io/projected/d326762b-a3ee-484d-89b2-47a64a7800c7-kube-api-access-l7w8z\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj\" (UID: \"d326762b-a3ee-484d-89b2-47a64a7800c7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:17.079155 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:17.079119    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:17.223953 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:17.223925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"]
Jul 27 17:34:17.227490 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:34:17.227465    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd326762b_a3ee_484d_89b2_47a64a7800c7.slice/crio-787b764c49ee87bad2b896eea7297d9a5e7603096c0445ce21dff3e04b90d394 WatchSource:0}: Error finding container 787b764c49ee87bad2b896eea7297d9a5e7603096c0445ce21dff3e04b90d394: Status 404 returned error can't find the container with id 787b764c49ee87bad2b896eea7297d9a5e7603096c0445ce21dff3e04b90d394
Jul 27 17:34:17.318584 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:17.318551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj" event={"ID":"d326762b-a3ee-484d-89b2-47a64a7800c7","Type":"ContainerStarted","Data":"787b764c49ee87bad2b896eea7297d9a5e7603096c0445ce21dff3e04b90d394"}
Jul 27 17:34:20.332249 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:20.332219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj" event={"ID":"d326762b-a3ee-484d-89b2-47a64a7800c7","Type":"ContainerStarted","Data":"e11b928f8e42f6f3cf373962bf73bf4de5c56f9215c08627d4b4f79e081e51eb"}
Jul 27 17:34:20.332583 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:20.332287    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:20.351735 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:20.351693    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj" podStartSLOduration=1.40746981 podStartE2EDuration="4.351671872s" totalImagesPullingTime="2.944202062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:34:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:34:20.349324618 +0000 UTC m=+418.060060283" watchObservedRunningTime="2026-07-27 17:34:20.351671872 +0000 UTC m=+418.062407538"
Jul 27 17:34:21.060477 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.060447    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"]
Jul 27 17:34:21.063662 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.063623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.065717 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.065694    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 27 17:34:21.065821 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.065724    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 27 17:34:21.065821 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.065736    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-2r9g4\""
Jul 27 17:34:21.074226 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.074207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"]
Jul 27 17:34:21.163551 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.163528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/94195308-5e01-418e-a3d5-c12632b08cb2-certificates\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.163660 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.163573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/94195308-5e01-418e-a3d5-c12632b08cb2-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.163660 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.163626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjlgn\" (UniqueName: \"kubernetes.io/projected/94195308-5e01-418e-a3d5-c12632b08cb2-kube-api-access-bjlgn\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.264161 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.264139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/94195308-5e01-418e-a3d5-c12632b08cb2-certificates\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.264250 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.264170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/94195308-5e01-418e-a3d5-c12632b08cb2-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.264250 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.264195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjlgn\" (UniqueName: \"kubernetes.io/projected/94195308-5e01-418e-a3d5-c12632b08cb2-kube-api-access-bjlgn\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.264579 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.264564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/94195308-5e01-418e-a3d5-c12632b08cb2-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.266809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.266783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/94195308-5e01-418e-a3d5-c12632b08cb2-certificates\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.272994 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.272975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjlgn\" (UniqueName: \"kubernetes.io/projected/94195308-5e01-418e-a3d5-c12632b08cb2-kube-api-access-bjlgn\") pod \"keda-metrics-apiserver-55557df8c-x2t8n\" (UID: \"94195308-5e01-418e-a3d5-c12632b08cb2\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.374465 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.374400    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:21.399184 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.399158    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-j8dxd"]
Jul 27 17:34:21.402477 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.402460    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.404506 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.404482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 27 17:34:21.411922 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.411896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-j8dxd"]
Jul 27 17:34:21.466644 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.465822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb4tt\" (UniqueName: \"kubernetes.io/projected/5c03958f-5986-4f10-90cc-5f954985847d-kube-api-access-sb4tt\") pod \"keda-admission-56f9798bb4-j8dxd\" (UID: \"5c03958f-5986-4f10-90cc-5f954985847d\") " pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.466644 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.465885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5c03958f-5986-4f10-90cc-5f954985847d-certificates\") pod \"keda-admission-56f9798bb4-j8dxd\" (UID: \"5c03958f-5986-4f10-90cc-5f954985847d\") " pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.501321 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.501295    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"]
Jul 27 17:34:21.503898 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:34:21.503861    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94195308_5e01_418e_a3d5_c12632b08cb2.slice/crio-dc05a0f4428f36c873ce67705399f8ec19a2cd262df0d6b09ff3721fc0be6941 WatchSource:0}: Error finding container dc05a0f4428f36c873ce67705399f8ec19a2cd262df0d6b09ff3721fc0be6941: Status 404 returned error can't find the container with id dc05a0f4428f36c873ce67705399f8ec19a2cd262df0d6b09ff3721fc0be6941
Jul 27 17:34:21.566745 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.566724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb4tt\" (UniqueName: \"kubernetes.io/projected/5c03958f-5986-4f10-90cc-5f954985847d-kube-api-access-sb4tt\") pod \"keda-admission-56f9798bb4-j8dxd\" (UID: \"5c03958f-5986-4f10-90cc-5f954985847d\") " pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.566849 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.566768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5c03958f-5986-4f10-90cc-5f954985847d-certificates\") pod \"keda-admission-56f9798bb4-j8dxd\" (UID: \"5c03958f-5986-4f10-90cc-5f954985847d\") " pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.569090 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.569072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5c03958f-5986-4f10-90cc-5f954985847d-certificates\") pod \"keda-admission-56f9798bb4-j8dxd\" (UID: \"5c03958f-5986-4f10-90cc-5f954985847d\") " pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.574417 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.574399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb4tt\" (UniqueName: \"kubernetes.io/projected/5c03958f-5986-4f10-90cc-5f954985847d-kube-api-access-sb4tt\") pod \"keda-admission-56f9798bb4-j8dxd\" (UID: \"5c03958f-5986-4f10-90cc-5f954985847d\") " pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.713267 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.713205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:21.833876 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:21.833848    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-j8dxd"]
Jul 27 17:34:21.836670 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:34:21.836644    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c03958f_5986_4f10_90cc_5f954985847d.slice/crio-c53690ac94fcbd5bd2aa911c985c260c2e86835375ecf08cf2f5ebf16e9b7cac WatchSource:0}: Error finding container c53690ac94fcbd5bd2aa911c985c260c2e86835375ecf08cf2f5ebf16e9b7cac: Status 404 returned error can't find the container with id c53690ac94fcbd5bd2aa911c985c260c2e86835375ecf08cf2f5ebf16e9b7cac
Jul 27 17:34:22.341930 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:22.341894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd" event={"ID":"5c03958f-5986-4f10-90cc-5f954985847d","Type":"ContainerStarted","Data":"c53690ac94fcbd5bd2aa911c985c260c2e86835375ecf08cf2f5ebf16e9b7cac"}
Jul 27 17:34:22.343216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:22.343189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n" event={"ID":"94195308-5e01-418e-a3d5-c12632b08cb2","Type":"ContainerStarted","Data":"dc05a0f4428f36c873ce67705399f8ec19a2cd262df0d6b09ff3721fc0be6941"}
Jul 27 17:34:25.358580 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:25.358539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd" event={"ID":"5c03958f-5986-4f10-90cc-5f954985847d","Type":"ContainerStarted","Data":"4e7e70907a7328fdba81360403258667f9d6003a4f75738c6b17493f40b0184b"}
Jul 27 17:34:25.358994 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:25.358642    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:34:25.359998 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:25.359975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n" event={"ID":"94195308-5e01-418e-a3d5-c12632b08cb2","Type":"ContainerStarted","Data":"a8361afd63e12ebd8bac5ff13d31ef29c694567ba7ca61118e958a7eaa11ccde"}
Jul 27 17:34:25.360114 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:25.360097    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:25.373486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:25.373444    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd" podStartSLOduration=1.073238294 podStartE2EDuration="4.37343137s" totalImagesPullingTime="3.300193076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:34:25.372719943 +0000 UTC m=+423.083455605" watchObservedRunningTime="2026-07-27 17:34:25.37343137 +0000 UTC m=+423.084167034"
Jul 27 17:34:25.387632 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:25.387562    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n" podStartSLOduration=0.747792707 podStartE2EDuration="4.387552126s" totalImagesPullingTime="3.639759419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:34:25.386320643 +0000 UTC m=+423.097056310" watchObservedRunningTime="2026-07-27 17:34:25.387552126 +0000 UTC m=+423.098287790"
Jul 27 17:34:36.368674 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:36.368645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-x2t8n"
Jul 27 17:34:41.338813 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:41.338778    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-7f9dj"
Jul 27 17:34:46.366885 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:34:46.366804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-j8dxd"
Jul 27 17:37:22.717739 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:37:22.717713    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:37:22.725746 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:37:22.725726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:38:40.477005 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.476962    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"]
Jul 27 17:38:40.480169 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.480153    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.482175 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.482130    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-24ce3-kube-rbac-proxy-sar-config\""
Jul 27 17:38:40.482357 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.482215    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 27 17:38:40.482440 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.482425    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-24ce3-predictor-serving-cert\""
Jul 27 17:38:40.482915 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.482899    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 27 17:38:40.482915 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.482912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-x6c7x\""
Jul 27 17:38:40.488720 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.488700    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"]
Jul 27 17:38:40.635685 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.635657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.635850 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.635692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.635850 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.635731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5gnt\" (UniqueName: \"kubernetes.io/projected/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-api-access-m5gnt\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.737028 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.736954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.737028 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.736987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.737028 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.737011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5gnt\" (UniqueName: \"kubernetes.io/projected/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-api-access-m5gnt\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.737281 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:38:40.737169    2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-serving-cert: secret "success-200-isvc-24ce3-predictor-serving-cert" not found
Jul 27 17:38:40.737281 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:38:40.737256    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls podName:b4a46e24-b4db-4556-bd25-0036d5c98d56 nodeName:}" failed. No retries permitted until 2026-07-27 17:38:41.237234087 +0000 UTC m=+678.947969743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls") pod "success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" (UID: "b4a46e24-b4db-4556-bd25-0036d5c98d56") : secret "success-200-isvc-24ce3-predictor-serving-cert" not found
Jul 27 17:38:40.737636 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.737616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.745418 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.745394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5gnt\" (UniqueName: \"kubernetes.io/projected/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-api-access-m5gnt\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:40.859178 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.859149    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"]
Jul 27 17:38:40.862491 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.862473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:40.864777 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.864753    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-24ce3-predictor-serving-cert\""
Jul 27 17:38:40.864868 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.864753    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-24ce3-kube-rbac-proxy-sar-config\""
Jul 27 17:38:40.872969 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.872946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"]
Jul 27 17:38:40.939033 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.939011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:40.939141 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.939043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5e3b758-5d89-482a-99fe-961f56e4a2a7-proxy-tls\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:40.939141 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:40.939085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xhmp\" (UniqueName: \"kubernetes.io/projected/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-api-access-2xhmp\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.039666 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.039636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.039790 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.039676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5e3b758-5d89-482a-99fe-961f56e4a2a7-proxy-tls\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.039790 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.039710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xhmp\" (UniqueName: \"kubernetes.io/projected/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-api-access-2xhmp\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.040248 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.040229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.042016 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.041994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5e3b758-5d89-482a-99fe-961f56e4a2a7-proxy-tls\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.047224 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.047202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xhmp\" (UniqueName: \"kubernetes.io/projected/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-api-access-2xhmp\") pod \"error-404-isvc-24ce3-predictor-565cbdc96-pwxvt\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") " pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.172462 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.172434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:41.240943 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.240913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:41.243038 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.243014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls\") pod \"success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") " pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:41.297739 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.297714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"]
Jul 27 17:38:41.300103 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:38:41.300013    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e3b758_5d89_482a_99fe_961f56e4a2a7.slice/crio-1d5f7a0a085f6cdb105c997cdc9e6d1151fa472aaed5c5d58dd6c1cb55b1740a WatchSource:0}: Error finding container 1d5f7a0a085f6cdb105c997cdc9e6d1151fa472aaed5c5d58dd6c1cb55b1740a: Status 404 returned error can't find the container with id 1d5f7a0a085f6cdb105c997cdc9e6d1151fa472aaed5c5d58dd6c1cb55b1740a
Jul 27 17:38:41.302047 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.302031    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:38:41.356074 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.356041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" event={"ID":"c5e3b758-5d89-482a-99fe-961f56e4a2a7","Type":"ContainerStarted","Data":"1d5f7a0a085f6cdb105c997cdc9e6d1151fa472aaed5c5d58dd6c1cb55b1740a"}
Jul 27 17:38:41.390505 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.390484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:41.513984 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:41.513947    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"]
Jul 27 17:38:41.516623 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:38:41.516597    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a46e24_b4db_4556_bd25_0036d5c98d56.slice/crio-331034c91c3db4ad64776beb38eb02d9bf8ddb707ca53911dc01a375cba5b78b WatchSource:0}: Error finding container 331034c91c3db4ad64776beb38eb02d9bf8ddb707ca53911dc01a375cba5b78b: Status 404 returned error can't find the container with id 331034c91c3db4ad64776beb38eb02d9bf8ddb707ca53911dc01a375cba5b78b
Jul 27 17:38:42.373475 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:42.373438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" event={"ID":"b4a46e24-b4db-4556-bd25-0036d5c98d56","Type":"ContainerStarted","Data":"331034c91c3db4ad64776beb38eb02d9bf8ddb707ca53911dc01a375cba5b78b"}
Jul 27 17:38:56.455972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:56.455875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" event={"ID":"c5e3b758-5d89-482a-99fe-961f56e4a2a7","Type":"ContainerStarted","Data":"a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22"}
Jul 27 17:38:56.457246 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:56.457218    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" event={"ID":"b4a46e24-b4db-4556-bd25-0036d5c98d56","Type":"ContainerStarted","Data":"3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9"}
Jul 27 17:38:59.473642 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.473603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" event={"ID":"c5e3b758-5d89-482a-99fe-961f56e4a2a7","Type":"ContainerStarted","Data":"45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87"}
Jul 27 17:38:59.474065 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.473674    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:59.474065 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.473704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:38:59.475405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.475381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" event={"ID":"b4a46e24-b4db-4556-bd25-0036d5c98d56","Type":"ContainerStarted","Data":"5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617"}
Jul 27 17:38:59.475405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.475386    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Jul 27 17:38:59.475545 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.475515    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:59.475575 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.475552    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:38:59.476522 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.476499    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 27 17:38:59.491570 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.491532    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podStartSLOduration=2.145148878 podStartE2EDuration="19.491521649s" totalImagesPullingTime="17.346372771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:38:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:38:59.489967044 +0000 UTC m=+697.200702709" watchObservedRunningTime="2026-07-27 17:38:59.491521649 +0000 UTC m=+697.202257326"
Jul 27 17:38:59.506540 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:38:59.506498    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podStartSLOduration=2.370896606 podStartE2EDuration="19.50648362s" totalImagesPullingTime="17.135587014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:38:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:38:59.505967066 +0000 UTC m=+697.216702734" watchObservedRunningTime="2026-07-27 17:38:59.50648362 +0000 UTC m=+697.217219287"
Jul 27 17:39:00.479298 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:00.479255    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 27 17:39:00.479682 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:00.479349    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Jul 27 17:39:05.484070 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:05.484037    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:39:05.484534 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:05.484519    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:39:05.484606 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:05.484582    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Jul 27 17:39:05.484940 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:05.484915    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 27 17:39:15.485115 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:15.485032    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 27 17:39:15.485115 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:15.485032    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Jul 27 17:39:25.485054 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:25.485015    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 27 17:39:25.485445 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:25.485023    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Jul 27 17:39:35.485280 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:35.485236    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Jul 27 17:39:35.485806 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:35.485236    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Jul 27 17:39:45.485075 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:45.485047    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:39:45.485549 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:39:45.485432    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:40:10.694105 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.694069    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"]
Jul 27 17:40:10.694638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.694318    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container" containerID="cri-o://3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9" gracePeriod=30
Jul 27 17:40:10.694638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.694419    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kube-rbac-proxy" containerID="cri-o://5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617" gracePeriod=30
Jul 27 17:40:10.749777 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.749748    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"]
Jul 27 17:40:10.753212 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.753196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:10.755467 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.755446    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-78126-predictor-serving-cert\""
Jul 27 17:40:10.755644 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.755629    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-78126-kube-rbac-proxy-sar-config\""
Jul 27 17:40:10.759015 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.758997    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"]
Jul 27 17:40:10.759190 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.759168    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container" containerID="cri-o://a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22" gracePeriod=30
Jul 27 17:40:10.759253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.759188    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kube-rbac-proxy" containerID="cri-o://45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87" gracePeriod=30
Jul 27 17:40:10.763458 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.763436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"]
Jul 27 17:40:10.861954 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.861928    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"]
Jul 27 17:40:10.865121 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.865098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:10.866868 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.866850    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-78126-predictor-serving-cert\""
Jul 27 17:40:10.866962 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.866901    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-78126-kube-rbac-proxy-sar-config\""
Jul 27 17:40:10.874345 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.874311    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"]
Jul 27 17:40:10.913954 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.913930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:10.914053 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.913968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccvn\" (UniqueName: \"kubernetes.io/projected/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-api-access-6ccvn\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:10.914053 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:10.913991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-proxy-tls\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.015218 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccvn\" (UniqueName: \"kubernetes.io/projected/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-api-access-6ccvn\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.015218 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-proxy-tls\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.015379 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ggl\" (UniqueName: \"kubernetes.io/projected/5ff180e4-f789-4010-8388-3974ef9f061c-kube-api-access-95ggl\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.015379 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.015483 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ff180e4-f789-4010-8388-3974ef9f061c-proxy-tls\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.015483 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ff180e4-f789-4010-8388-3974ef9f061c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.015950 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.015929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.017503 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.017488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-proxy-tls\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.021778 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.021757    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccvn\" (UniqueName: \"kubernetes.io/projected/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-api-access-6ccvn\") pod \"success-200-isvc-78126-predictor-5c95549787-2v6h5\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") " pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.064144 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.064120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.116836 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.116811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ggl\" (UniqueName: \"kubernetes.io/projected/5ff180e4-f789-4010-8388-3974ef9f061c-kube-api-access-95ggl\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.116954 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.116858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ff180e4-f789-4010-8388-3974ef9f061c-proxy-tls\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.116954 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.116880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ff180e4-f789-4010-8388-3974ef9f061c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.117528 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.117500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ff180e4-f789-4010-8388-3974ef9f061c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.119556 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.119519    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ff180e4-f789-4010-8388-3974ef9f061c-proxy-tls\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.127233 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.127212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ggl\" (UniqueName: \"kubernetes.io/projected/5ff180e4-f789-4010-8388-3974ef9f061c-kube-api-access-95ggl\") pod \"error-404-isvc-78126-predictor-7fbf689c8b-ctxpl\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") " pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.176132 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.176106    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.194691 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.194669    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"]
Jul 27 17:40:11.197534 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:40:11.197504    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa91a475_0b92_48f1_8c4d_ec181f9fcac0.slice/crio-21c0a7579da98f751c70a6946e10bf377c6c25108abc021bd836ba3f22a6f6ee WatchSource:0}: Error finding container 21c0a7579da98f751c70a6946e10bf377c6c25108abc021bd836ba3f22a6f6ee: Status 404 returned error can't find the container with id 21c0a7579da98f751c70a6946e10bf377c6c25108abc021bd836ba3f22a6f6ee
Jul 27 17:40:11.309053 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.309032    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"]
Jul 27 17:40:11.311060 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:40:11.311037    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff180e4_f789_4010_8388_3974ef9f061c.slice/crio-8e8afd0ef023854a9e65d49476f6aeb4901c07c1aecd52536a2083f34085a57e WatchSource:0}: Error finding container 8e8afd0ef023854a9e65d49476f6aeb4901c07c1aecd52536a2083f34085a57e: Status 404 returned error can't find the container with id 8e8afd0ef023854a9e65d49476f6aeb4901c07c1aecd52536a2083f34085a57e
Jul 27 17:40:11.768312 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.768274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" event={"ID":"fa91a475-0b92-48f1-8c4d-ec181f9fcac0","Type":"ContainerStarted","Data":"bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3"}
Jul 27 17:40:11.768732 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.768398    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.768732 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.768435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" event={"ID":"fa91a475-0b92-48f1-8c4d-ec181f9fcac0","Type":"ContainerStarted","Data":"b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9"}
Jul 27 17:40:11.768732 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.768473    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:11.768732 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.768499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" event={"ID":"fa91a475-0b92-48f1-8c4d-ec181f9fcac0","Type":"ContainerStarted","Data":"21c0a7579da98f751c70a6946e10bf377c6c25108abc021bd836ba3f22a6f6ee"}
Jul 27 17:40:11.769775 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.769744    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:40:11.770005 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.769979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" event={"ID":"5ff180e4-f789-4010-8388-3974ef9f061c","Type":"ContainerStarted","Data":"e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76"}
Jul 27 17:40:11.770074 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.770036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" event={"ID":"5ff180e4-f789-4010-8388-3974ef9f061c","Type":"ContainerStarted","Data":"0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0"}
Jul 27 17:40:11.770074 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.770064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.770165 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.770083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" event={"ID":"5ff180e4-f789-4010-8388-3974ef9f061c","Type":"ContainerStarted","Data":"8e8afd0ef023854a9e65d49476f6aeb4901c07c1aecd52536a2083f34085a57e"}
Jul 27 17:40:11.770165 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.770133    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:11.771449 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.771425    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:40:11.771690 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.771669    2563 generic.go:356] "Generic (PLEG): container finished" podID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerID="45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87" exitCode=2
Jul 27 17:40:11.771754 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.771737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" event={"ID":"c5e3b758-5d89-482a-99fe-961f56e4a2a7","Type":"ContainerDied","Data":"45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87"}
Jul 27 17:40:11.773214 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.773195    2563 generic.go:356] "Generic (PLEG): container finished" podID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerID="5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617" exitCode=2
Jul 27 17:40:11.773293 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.773221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" event={"ID":"b4a46e24-b4db-4556-bd25-0036d5c98d56","Type":"ContainerDied","Data":"5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617"}
Jul 27 17:40:11.787877 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.787838    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podStartSLOduration=1.787826049 podStartE2EDuration="1.787826049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:40:11.785970035 +0000 UTC m=+769.496705715" watchObservedRunningTime="2026-07-27 17:40:11.787826049 +0000 UTC m=+769.498561715"
Jul 27 17:40:11.803282 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:11.803243    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podStartSLOduration=1.803232182 podStartE2EDuration="1.803232182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:40:11.802118107 +0000 UTC m=+769.512853784" watchObservedRunningTime="2026-07-27 17:40:11.803232182 +0000 UTC m=+769.513967846"
Jul 27 17:40:12.778516 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:12.778479    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:40:12.778965 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:12.778574    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:40:13.555581 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.555561    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:40:13.558041 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.558023    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:40:13.741057 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.740995    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-rbac-proxy-sar-config\") pod \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") "
Jul 27 17:40:13.741057 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-rbac-proxy-sar-config\") pod \"b4a46e24-b4db-4556-bd25-0036d5c98d56\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") "
Jul 27 17:40:13.741216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741060    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xhmp\" (UniqueName: \"kubernetes.io/projected/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-api-access-2xhmp\") pod \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") "
Jul 27 17:40:13.741216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5gnt\" (UniqueName: \"kubernetes.io/projected/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-api-access-m5gnt\") pod \"b4a46e24-b4db-4556-bd25-0036d5c98d56\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") "
Jul 27 17:40:13.741216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741132    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5e3b758-5d89-482a-99fe-961f56e4a2a7-proxy-tls\") pod \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\" (UID: \"c5e3b758-5d89-482a-99fe-961f56e4a2a7\") "
Jul 27 17:40:13.741216 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741165    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls\") pod \"b4a46e24-b4db-4556-bd25-0036d5c98d56\" (UID: \"b4a46e24-b4db-4556-bd25-0036d5c98d56\") "
Jul 27 17:40:13.741497 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741423    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c5e3b758-5d89-482a-99fe-961f56e4a2a7" (UID: "c5e3b758-5d89-482a-99fe-961f56e4a2a7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:40:13.741497 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.741477    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b4a46e24-b4db-4556-bd25-0036d5c98d56" (UID: "b4a46e24-b4db-4556-bd25-0036d5c98d56"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:40:13.743286 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.743259    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-api-access-2xhmp" (OuterVolumeSpecName: "kube-api-access-2xhmp") pod "c5e3b758-5d89-482a-99fe-961f56e4a2a7" (UID: "c5e3b758-5d89-482a-99fe-961f56e4a2a7"). InnerVolumeSpecName "kube-api-access-2xhmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:40:13.743400 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.743313    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-api-access-m5gnt" (OuterVolumeSpecName: "kube-api-access-m5gnt") pod "b4a46e24-b4db-4556-bd25-0036d5c98d56" (UID: "b4a46e24-b4db-4556-bd25-0036d5c98d56"). InnerVolumeSpecName "kube-api-access-m5gnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:40:13.743400 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.743323    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b4a46e24-b4db-4556-bd25-0036d5c98d56" (UID: "b4a46e24-b4db-4556-bd25-0036d5c98d56"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:40:13.743400 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.743313    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5e3b758-5d89-482a-99fe-961f56e4a2a7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c5e3b758-5d89-482a-99fe-961f56e4a2a7" (UID: "c5e3b758-5d89-482a-99fe-961f56e4a2a7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:40:13.785168 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.785143    2563 generic.go:356] "Generic (PLEG): container finished" podID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerID="a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22" exitCode=0
Jul 27 17:40:13.785519 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.785199    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"
Jul 27 17:40:13.785519 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.785230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" event={"ID":"c5e3b758-5d89-482a-99fe-961f56e4a2a7","Type":"ContainerDied","Data":"a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22"}
Jul 27 17:40:13.785519 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.785302    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt" event={"ID":"c5e3b758-5d89-482a-99fe-961f56e4a2a7","Type":"ContainerDied","Data":"1d5f7a0a085f6cdb105c997cdc9e6d1151fa472aaed5c5d58dd6c1cb55b1740a"}
Jul 27 17:40:13.785519 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.785321    2563 scope.go:117] "RemoveContainer" containerID="45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87"
Jul 27 17:40:13.786960 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.786936    2563 generic.go:356] "Generic (PLEG): container finished" podID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerID="3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9" exitCode=0
Jul 27 17:40:13.787043 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.786975    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"
Jul 27 17:40:13.787043 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.787024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" event={"ID":"b4a46e24-b4db-4556-bd25-0036d5c98d56","Type":"ContainerDied","Data":"3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9"}
Jul 27 17:40:13.787121 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.787084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6" event={"ID":"b4a46e24-b4db-4556-bd25-0036d5c98d56","Type":"ContainerDied","Data":"331034c91c3db4ad64776beb38eb02d9bf8ddb707ca53911dc01a375cba5b78b"}
Jul 27 17:40:13.793751 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.793732    2563 scope.go:117] "RemoveContainer" containerID="a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22"
Jul 27 17:40:13.800962 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.800937    2563 scope.go:117] "RemoveContainer" containerID="45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87"
Jul 27 17:40:13.801182 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:40:13.801164    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87\": container with ID starting with 45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87 not found: ID does not exist" containerID="45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87"
Jul 27 17:40:13.801237 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.801189    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87"} err="failed to get container status \"45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87\": rpc error: code = NotFound desc = could not find container \"45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87\": container with ID starting with 45873163aa886368bed7fd6693a5b5ff3da0ecfb075ef2031f16e58069d9ef87 not found: ID does not exist"
Jul 27 17:40:13.801237 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.801205    2563 scope.go:117] "RemoveContainer" containerID="a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22"
Jul 27 17:40:13.801445 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:40:13.801427    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22\": container with ID starting with a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22 not found: ID does not exist" containerID="a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22"
Jul 27 17:40:13.801488 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.801451    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22"} err="failed to get container status \"a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22\": rpc error: code = NotFound desc = could not find container \"a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22\": container with ID starting with a9e0c5ecd0a3edfa64f1c4a1b4c0eec22b7aa41bc5cb467bdc55b3d6958d4c22 not found: ID does not exist"
Jul 27 17:40:13.801488 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.801467    2563 scope.go:117] "RemoveContainer" containerID="5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617"
Jul 27 17:40:13.806762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.806742    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"]
Jul 27 17:40:13.809318 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.809300    2563 scope.go:117] "RemoveContainer" containerID="3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9"
Jul 27 17:40:13.810261 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.810169    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-24ce3-predictor-565cbdc96-pwxvt"]
Jul 27 17:40:13.815984 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.815968    2563 scope.go:117] "RemoveContainer" containerID="5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617"
Jul 27 17:40:13.816227 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:40:13.816211    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617\": container with ID starting with 5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617 not found: ID does not exist" containerID="5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617"
Jul 27 17:40:13.816272 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.816236    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617"} err="failed to get container status \"5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617\": rpc error: code = NotFound desc = could not find container \"5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617\": container with ID starting with 5481023127c6330498c8c4bbe87741fde75b4013e6f8bcadcc4c3c3b75198617 not found: ID does not exist"
Jul 27 17:40:13.816272 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.816252    2563 scope.go:117] "RemoveContainer" containerID="3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9"
Jul 27 17:40:13.816496 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:40:13.816477    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9\": container with ID starting with 3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9 not found: ID does not exist" containerID="3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9"
Jul 27 17:40:13.816541 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.816500    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9"} err="failed to get container status \"3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9\": rpc error: code = NotFound desc = could not find container \"3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9\": container with ID starting with 3c862c064b8c84e24ff8de1acc7bb5f538757b3655b1e7ee82cf156535a11be9 not found: ID does not exist"
Jul 27 17:40:13.818665 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.818646    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"]
Jul 27 17:40:13.820987 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.820968    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-24ce3-predictor-6dd44d8cd5-kxqt6"]
Jul 27 17:40:13.841860 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.841839    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xhmp\" (UniqueName: \"kubernetes.io/projected/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-api-access-2xhmp\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:40:13.841860 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.841858    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5gnt\" (UniqueName: \"kubernetes.io/projected/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-api-access-m5gnt\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:40:13.841968 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.841869    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5e3b758-5d89-482a-99fe-961f56e4a2a7-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:40:13.841968 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.841878    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4a46e24-b4db-4556-bd25-0036d5c98d56-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:40:13.841968 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.841885    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5e3b758-5d89-482a-99fe-961f56e4a2a7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:40:13.841968 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:13.841893    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4a46e24-b4db-4556-bd25-0036d5c98d56-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:40:14.799228 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:14.799194    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" path="/var/lib/kubelet/pods/b4a46e24-b4db-4556-bd25-0036d5c98d56/volumes"
Jul 27 17:40:14.799616 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:14.799602    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" path="/var/lib/kubelet/pods/c5e3b758-5d89-482a-99fe-961f56e4a2a7/volumes"
Jul 27 17:40:17.782258 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:17.782231    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:17.782732 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:17.782597    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:40:17.782775 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:17.782727    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:40:17.782967 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:17.782946    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:40:27.783299 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:27.783263    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:40:27.783685 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:27.783259    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:40:37.782770 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:37.782736    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:40:37.783237 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:37.782983    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:40:47.782812 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:47.782723    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:40:47.783172 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:47.782945    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:40:57.783152 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:57.783122    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:40:57.783839 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:40:57.783819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:41:10.798105 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"]
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798540    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kube-rbac-proxy"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798561    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kube-rbac-proxy"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798581    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kube-rbac-proxy"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798589    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kube-rbac-proxy"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798609    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798618    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798628    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container"
Jul 27 17:41:10.798664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798636    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container"
Jul 27 17:41:10.799036 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798702    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kserve-container"
Jul 27 17:41:10.799036 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798716    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kube-rbac-proxy"
Jul 27 17:41:10.799036 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798728    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e3b758-5d89-482a-99fe-961f56e4a2a7" containerName="kube-rbac-proxy"
Jul 27 17:41:10.799036 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.798738    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a46e24-b4db-4556-bd25-0036d5c98d56" containerName="kserve-container"
Jul 27 17:41:10.801935 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.801912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:10.805120 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.805101    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-796c8-predictor-serving-cert\""
Jul 27 17:41:10.805427 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.805407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-796c8-kube-rbac-proxy-sar-config\""
Jul 27 17:41:10.815485 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.815466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"]
Jul 27 17:41:10.913064 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.913038    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"]
Jul 27 17:41:10.916404 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.916389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:10.918435 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.918416    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-796c8-kube-rbac-proxy-sar-config\""
Jul 27 17:41:10.918581 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.918443    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-796c8-predictor-serving-cert\""
Jul 27 17:41:10.928624 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.928604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"]
Jul 27 17:41:10.950119 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.950090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:10.950226 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.950132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbdtm\" (UniqueName: \"kubernetes.io/projected/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-api-access-xbdtm\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:10.950226 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:10.950219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.050809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.050751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.050809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.050792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.050943 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.050820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.050943 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:41:11.050905    2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-796c8-predictor-serving-cert: secret "success-200-isvc-796c8-predictor-serving-cert" not found
Jul 27 17:41:11.051037 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.050940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xc8m\" (UniqueName: \"kubernetes.io/projected/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-api-access-5xc8m\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.051037 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:41:11.050986    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls podName:7d32c32b-f978-4bf7-8ccf-6379858b4821 nodeName:}" failed. No retries permitted until 2026-07-27 17:41:11.550968063 +0000 UTC m=+829.261703707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls") pod "success-200-isvc-796c8-predictor-79d76b5df7-q46rf" (UID: "7d32c32b-f978-4bf7-8ccf-6379858b4821") : secret "success-200-isvc-796c8-predictor-serving-cert" not found
Jul 27 17:41:11.051037 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.051000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbdtm\" (UniqueName: \"kubernetes.io/projected/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-api-access-xbdtm\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.051037 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.051036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2141d288-3ec1-4fb9-811d-598a3066e8ea-proxy-tls\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.051383 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.051367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.061942 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.061925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbdtm\" (UniqueName: \"kubernetes.io/projected/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-api-access-xbdtm\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.152172 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.152142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xc8m\" (UniqueName: \"kubernetes.io/projected/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-api-access-5xc8m\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.152271 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.152198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2141d288-3ec1-4fb9-811d-598a3066e8ea-proxy-tls\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.152359 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.152279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.152839 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.152818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.154346 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.154316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2141d288-3ec1-4fb9-811d-598a3066e8ea-proxy-tls\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.160607 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.160585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xc8m\" (UniqueName: \"kubernetes.io/projected/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-api-access-5xc8m\") pod \"error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") " pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.226059 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.226035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:11.355092 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.355067    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"]
Jul 27 17:41:11.357105 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:41:11.357081    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2141d288_3ec1_4fb9_811d_598a3066e8ea.slice/crio-029e8957808227e495758841f00a5ec0447bc55b42db6838f40a05ef40a6d270 WatchSource:0}: Error finding container 029e8957808227e495758841f00a5ec0447bc55b42db6838f40a05ef40a6d270: Status 404 returned error can't find the container with id 029e8957808227e495758841f00a5ec0447bc55b42db6838f40a05ef40a6d270
Jul 27 17:41:11.555375 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.555299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.558149 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.557861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls\") pod \"success-200-isvc-796c8-predictor-79d76b5df7-q46rf\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") " pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.712467 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.712442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:11.834294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:11.834268    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"]
Jul 27 17:41:11.836188 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:41:11.836158    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d32c32b_f978_4bf7_8ccf_6379858b4821.slice/crio-8bb1b31f91ca1b75d99034cd8f9686fe16420aad054253e7b2531f9e1812160e WatchSource:0}: Error finding container 8bb1b31f91ca1b75d99034cd8f9686fe16420aad054253e7b2531f9e1812160e: Status 404 returned error can't find the container with id 8bb1b31f91ca1b75d99034cd8f9686fe16420aad054253e7b2531f9e1812160e
Jul 27 17:41:12.020903 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.020869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" event={"ID":"7d32c32b-f978-4bf7-8ccf-6379858b4821","Type":"ContainerStarted","Data":"65ea9bf824df8a8a4ef8801b35ba916cf2feb8ff17387b9b5558486f7cb7ddff"}
Jul 27 17:41:12.021053 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.020928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" event={"ID":"7d32c32b-f978-4bf7-8ccf-6379858b4821","Type":"ContainerStarted","Data":"90ffbc56bb71671e74a5ce557aa4179cce4d16a5ba09f6e7e15f424707f8d914"}
Jul 27 17:41:12.021053 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.020954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" event={"ID":"7d32c32b-f978-4bf7-8ccf-6379858b4821","Type":"ContainerStarted","Data":"8bb1b31f91ca1b75d99034cd8f9686fe16420aad054253e7b2531f9e1812160e"}
Jul 27 17:41:12.021053 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.020985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:12.021175 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.021151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:12.022648 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.022626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" event={"ID":"2141d288-3ec1-4fb9-811d-598a3066e8ea","Type":"ContainerStarted","Data":"8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c"}
Jul 27 17:41:12.022745 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.022666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" event={"ID":"2141d288-3ec1-4fb9-811d-598a3066e8ea","Type":"ContainerStarted","Data":"0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674"}
Jul 27 17:41:12.022745 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.022671    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:41:12.022745 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.022689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" event={"ID":"2141d288-3ec1-4fb9-811d-598a3066e8ea","Type":"ContainerStarted","Data":"029e8957808227e495758841f00a5ec0447bc55b42db6838f40a05ef40a6d270"}
Jul 27 17:41:12.022852 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.022778    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:12.022852 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.022813    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:12.023845 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.023822    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:41:12.040920 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.040882    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podStartSLOduration=2.040871752 podStartE2EDuration="2.040871752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:41:12.03997633 +0000 UTC m=+829.750711996" watchObservedRunningTime="2026-07-27 17:41:12.040871752 +0000 UTC m=+829.751607493"
Jul 27 17:41:12.059808 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:12.059756    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podStartSLOduration=2.059743907 podStartE2EDuration="2.059743907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:41:12.058659864 +0000 UTC m=+829.769395528" watchObservedRunningTime="2026-07-27 17:41:12.059743907 +0000 UTC m=+829.770479572"
Jul 27 17:41:13.026773 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:13.026731    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:41:13.027202 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:13.026880    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:41:18.030960 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:18.030922    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:41:18.031362 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:18.031199    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:18.031409 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:18.031355    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:41:18.031762 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:18.031728    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:41:28.032244 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:28.032209    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:41:28.032638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:28.032209    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:41:38.031775 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:38.031733    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:41:38.032144 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:38.031734    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:41:48.031628 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:48.031591    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:41:48.032075 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:48.031631    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:41:58.032295 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:58.032268    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:41:58.032683 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:41:58.032359    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:42:22.746930 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:42:22.746823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:42:22.752657 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:42:22.752638    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:47:22.774129 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:47:22.774052    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:47:22.777365 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:47:22.777344    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:49:25.556032 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.555998    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"]
Jul 27 17:49:25.558602 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.556275    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" containerID="cri-o://b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9" gracePeriod=30
Jul 27 17:49:25.558602 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.556325    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kube-rbac-proxy" containerID="cri-o://bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3" gracePeriod=30
Jul 27 17:49:25.617696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.617672    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"]
Jul 27 17:49:25.620980 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.620963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.622833 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.622812    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-37f6e-kube-rbac-proxy-sar-config\""
Jul 27 17:49:25.622916 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.622902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-37f6e-predictor-serving-cert\""
Jul 27 17:49:25.630668 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.630650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"]
Jul 27 17:49:25.696606 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.696583    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"]
Jul 27 17:49:25.699848 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.699833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.701625 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.701608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-37f6e-predictor-serving-cert\""
Jul 27 17:49:25.701713 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.701647    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-37f6e-kube-rbac-proxy-sar-config\""
Jul 27 17:49:25.708178 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.708137    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"]
Jul 27 17:49:25.717021 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.716993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.717143 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.717123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-api-access-5wxll\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.717204 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.717193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.741893 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.741869    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"]
Jul 27 17:49:25.742077 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.742054    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" containerID="cri-o://0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0" gracePeriod=30
Jul 27 17:49:25.742127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.742085    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kube-rbac-proxy" containerID="cri-o://e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76" gracePeriod=30
Jul 27 17:49:25.817977 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.817930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.818061 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.817977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.818061 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.818011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc5bb\" (UniqueName: \"kubernetes.io/projected/592f360c-7d03-4896-90eb-abb9af963334-kube-api-access-sc5bb\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.818061 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.818036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.818061 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.818052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/592f360c-7d03-4896-90eb-abb9af963334-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.818188 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:25.818071    2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-serving-cert: secret "success-200-isvc-37f6e-predictor-serving-cert" not found
Jul 27 17:49:25.818188 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:25.818134    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls podName:af339b57-0fcc-45f6-9452-7d294ba8ced2 nodeName:}" failed. No retries permitted until 2026-07-27 17:49:26.318118185 +0000 UTC m=+1324.028853834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls") pod "success-200-isvc-37f6e-predictor-54465b587c-v9rfx" (UID: "af339b57-0fcc-45f6-9452-7d294ba8ced2") : secret "success-200-isvc-37f6e-predictor-serving-cert" not found
Jul 27 17:49:25.818188 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.818152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-api-access-5wxll\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.818648 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.818630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.828249 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.828231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-api-access-5wxll\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:25.918841 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.918822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc5bb\" (UniqueName: \"kubernetes.io/projected/592f360c-7d03-4896-90eb-abb9af963334-kube-api-access-sc5bb\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.918937 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.918857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.918937 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.918876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/592f360c-7d03-4896-90eb-abb9af963334-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.919042 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:25.918972    2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-serving-cert: secret "error-404-isvc-37f6e-predictor-serving-cert" not found
Jul 27 17:49:25.919042 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:25.919033    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls podName:592f360c-7d03-4896-90eb-abb9af963334 nodeName:}" failed. No retries permitted until 2026-07-27 17:49:26.419018347 +0000 UTC m=+1324.129753992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls") pod "error-404-isvc-37f6e-predictor-848b7c596-c679g" (UID: "592f360c-7d03-4896-90eb-abb9af963334") : secret "error-404-isvc-37f6e-predictor-serving-cert" not found
Jul 27 17:49:25.919557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.919534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/592f360c-7d03-4896-90eb-abb9af963334-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:25.928232 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:25.928210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc5bb\" (UniqueName: \"kubernetes.io/projected/592f360c-7d03-4896-90eb-abb9af963334-kube-api-access-sc5bb\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:26.110973 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.110899    2563 generic.go:356] "Generic (PLEG): container finished" podID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerID="bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3" exitCode=2
Jul 27 17:49:26.111100 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.110973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" event={"ID":"fa91a475-0b92-48f1-8c4d-ec181f9fcac0","Type":"ContainerDied","Data":"bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3"}
Jul 27 17:49:26.112558 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.112536    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ff180e4-f789-4010-8388-3974ef9f061c" containerID="e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76" exitCode=2
Jul 27 17:49:26.112651 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.112590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" event={"ID":"5ff180e4-f789-4010-8388-3974ef9f061c","Type":"ContainerDied","Data":"e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76"}
Jul 27 17:49:26.322152 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.322128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:26.324549 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.324519    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls\") pod \"success-200-isvc-37f6e-predictor-54465b587c-v9rfx\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") " pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:26.423316 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.423259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:26.425414 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.425396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls\") pod \"error-404-isvc-37f6e-predictor-848b7c596-c679g\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") " pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:26.530983 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.530959    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:26.610844 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.610506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:26.662496 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.662457    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"]
Jul 27 17:49:26.664945 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:49:26.664913    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf339b57_0fcc_45f6_9452_7d294ba8ced2.slice/crio-fdcba921815f581998791b8069f88fffc8630e47c0b16f8351262a3a7c8a1194 WatchSource:0}: Error finding container fdcba921815f581998791b8069f88fffc8630e47c0b16f8351262a3a7c8a1194: Status 404 returned error can't find the container with id fdcba921815f581998791b8069f88fffc8630e47c0b16f8351262a3a7c8a1194
Jul 27 17:49:26.744384 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:26.744363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"]
Jul 27 17:49:26.746734 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:49:26.746708    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592f360c_7d03_4896_90eb_abb9af963334.slice/crio-9bc272417e4cf2c6193cb24773838f99b20cdcac17da0fc87c0e621bf6121908 WatchSource:0}: Error finding container 9bc272417e4cf2c6193cb24773838f99b20cdcac17da0fc87c0e621bf6121908: Status 404 returned error can't find the container with id 9bc272417e4cf2c6193cb24773838f99b20cdcac17da0fc87c0e621bf6121908
Jul 27 17:49:27.119250 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.119207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" event={"ID":"592f360c-7d03-4896-90eb-abb9af963334","Type":"ContainerStarted","Data":"37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8"}
Jul 27 17:49:27.119446 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.119281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" event={"ID":"592f360c-7d03-4896-90eb-abb9af963334","Type":"ContainerStarted","Data":"3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e"}
Jul 27 17:49:27.119446 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.119313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" event={"ID":"592f360c-7d03-4896-90eb-abb9af963334","Type":"ContainerStarted","Data":"9bc272417e4cf2c6193cb24773838f99b20cdcac17da0fc87c0e621bf6121908"}
Jul 27 17:49:27.119446 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.119362    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:27.119446 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.119395    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:27.121066 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.121037    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 27 17:49:27.121171 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.121066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" event={"ID":"af339b57-0fcc-45f6-9452-7d294ba8ced2","Type":"ContainerStarted","Data":"5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535"}
Jul 27 17:49:27.121171 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.121117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" event={"ID":"af339b57-0fcc-45f6-9452-7d294ba8ced2","Type":"ContainerStarted","Data":"a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3"}
Jul 27 17:49:27.121171 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.121140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" event={"ID":"af339b57-0fcc-45f6-9452-7d294ba8ced2","Type":"ContainerStarted","Data":"fdcba921815f581998791b8069f88fffc8630e47c0b16f8351262a3a7c8a1194"}
Jul 27 17:49:27.121171 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.121162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:27.121308 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.121202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:27.122390 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.122367    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:49:27.136183 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.136143    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podStartSLOduration=2.136131298 podStartE2EDuration="2.136131298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:49:27.134084396 +0000 UTC m=+1324.844820062" watchObservedRunningTime="2026-07-27 17:49:27.136131298 +0000 UTC m=+1324.846866963"
Jul 27 17:49:27.155156 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.155116    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podStartSLOduration=2.155104017 podStartE2EDuration="2.155104017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:49:27.154540896 +0000 UTC m=+1324.865276562" watchObservedRunningTime="2026-07-27 17:49:27.155104017 +0000 UTC m=+1324.865839683"
Jul 27 17:49:27.779679 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.779640    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.29:8643/healthz\": dial tcp 10.132.0.29:8643: connect: connection refused"
Jul 27 17:49:27.780031 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.779640    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused"
Jul 27 17:49:27.782930 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.782905    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Jul 27 17:49:27.783055 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:27.783039    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Jul 27 17:49:28.125680 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.125585    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:49:28.125680 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.125670    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 27 17:49:28.403702 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.403675    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:49:28.540535 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.540509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ccvn\" (UniqueName: \"kubernetes.io/projected/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-api-access-6ccvn\") pod \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") "
Jul 27 17:49:28.540639 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.540549    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-proxy-tls\") pod \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") "
Jul 27 17:49:28.540639 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.540575    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-rbac-proxy-sar-config\") pod \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\" (UID: \"fa91a475-0b92-48f1-8c4d-ec181f9fcac0\") "
Jul 27 17:49:28.541008 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.540979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fa91a475-0b92-48f1-8c4d-ec181f9fcac0" (UID: "fa91a475-0b92-48f1-8c4d-ec181f9fcac0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:49:28.542536 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.542515    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fa91a475-0b92-48f1-8c4d-ec181f9fcac0" (UID: "fa91a475-0b92-48f1-8c4d-ec181f9fcac0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:49:28.542617 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.542594    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-api-access-6ccvn" (OuterVolumeSpecName: "kube-api-access-6ccvn") pod "fa91a475-0b92-48f1-8c4d-ec181f9fcac0" (UID: "fa91a475-0b92-48f1-8c4d-ec181f9fcac0"). InnerVolumeSpecName "kube-api-access-6ccvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:49:28.642124 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.642059    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ccvn\" (UniqueName: \"kubernetes.io/projected/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-api-access-6ccvn\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.642124 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.642086    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.642124 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.642096    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa91a475-0b92-48f1-8c4d-ec181f9fcac0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.685997 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.685976    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:49:28.844252 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.844222    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ff180e4-f789-4010-8388-3974ef9f061c-proxy-tls\") pod \"5ff180e4-f789-4010-8388-3974ef9f061c\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") "
Jul 27 17:49:28.844666 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.844291    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95ggl\" (UniqueName: \"kubernetes.io/projected/5ff180e4-f789-4010-8388-3974ef9f061c-kube-api-access-95ggl\") pod \"5ff180e4-f789-4010-8388-3974ef9f061c\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") "
Jul 27 17:49:28.844666 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.844368    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ff180e4-f789-4010-8388-3974ef9f061c-kube-rbac-proxy-sar-config\") pod \"5ff180e4-f789-4010-8388-3974ef9f061c\" (UID: \"5ff180e4-f789-4010-8388-3974ef9f061c\") "
Jul 27 17:49:28.844753 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.844673    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff180e4-f789-4010-8388-3974ef9f061c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5ff180e4-f789-4010-8388-3974ef9f061c" (UID: "5ff180e4-f789-4010-8388-3974ef9f061c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:49:28.846350 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.846315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff180e4-f789-4010-8388-3974ef9f061c-kube-api-access-95ggl" (OuterVolumeSpecName: "kube-api-access-95ggl") pod "5ff180e4-f789-4010-8388-3974ef9f061c" (UID: "5ff180e4-f789-4010-8388-3974ef9f061c"). InnerVolumeSpecName "kube-api-access-95ggl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:49:28.846789 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.846763    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff180e4-f789-4010-8388-3974ef9f061c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5ff180e4-f789-4010-8388-3974ef9f061c" (UID: "5ff180e4-f789-4010-8388-3974ef9f061c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:49:28.945809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.945745    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95ggl\" (UniqueName: \"kubernetes.io/projected/5ff180e4-f789-4010-8388-3974ef9f061c-kube-api-access-95ggl\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.945809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.945775    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ff180e4-f789-4010-8388-3974ef9f061c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.945809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:28.945789    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ff180e4-f789-4010-8388-3974ef9f061c-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:49:29.132622 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.132596    2563 generic.go:356] "Generic (PLEG): container finished" podID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerID="b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9" exitCode=0
Jul 27 17:49:29.132730 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.132663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"
Jul 27 17:49:29.132730 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.132679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" event={"ID":"fa91a475-0b92-48f1-8c4d-ec181f9fcac0","Type":"ContainerDied","Data":"b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9"}
Jul 27 17:49:29.132832 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.132749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5" event={"ID":"fa91a475-0b92-48f1-8c4d-ec181f9fcac0","Type":"ContainerDied","Data":"21c0a7579da98f751c70a6946e10bf377c6c25108abc021bd836ba3f22a6f6ee"}
Jul 27 17:49:29.132832 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.132786    2563 scope.go:117] "RemoveContainer" containerID="bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3"
Jul 27 17:49:29.134457 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.134437    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ff180e4-f789-4010-8388-3974ef9f061c" containerID="0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0" exitCode=0
Jul 27 17:49:29.134550 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.134476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"
Jul 27 17:49:29.134550 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.134495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" event={"ID":"5ff180e4-f789-4010-8388-3974ef9f061c","Type":"ContainerDied","Data":"0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0"}
Jul 27 17:49:29.134550 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.134536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl" event={"ID":"5ff180e4-f789-4010-8388-3974ef9f061c","Type":"ContainerDied","Data":"8e8afd0ef023854a9e65d49476f6aeb4901c07c1aecd52536a2083f34085a57e"}
Jul 27 17:49:29.142623 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.142607    2563 scope.go:117] "RemoveContainer" containerID="b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9"
Jul 27 17:49:29.150039 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.150023    2563 scope.go:117] "RemoveContainer" containerID="bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3"
Jul 27 17:49:29.150281 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:29.150262    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3\": container with ID starting with bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3 not found: ID does not exist" containerID="bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3"
Jul 27 17:49:29.150455 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.150292    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3"} err="failed to get container status \"bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3\": rpc error: code = NotFound desc = could not find container \"bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3\": container with ID starting with bd4d763a13c4e1075a3f78bd004592c52858b7db66d85819275c8622a09166b3 not found: ID does not exist"
Jul 27 17:49:29.150455 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.150313    2563 scope.go:117] "RemoveContainer" containerID="b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9"
Jul 27 17:49:29.150636 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:29.150616    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9\": container with ID starting with b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9 not found: ID does not exist" containerID="b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9"
Jul 27 17:49:29.150702 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.150646    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9"} err="failed to get container status \"b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9\": rpc error: code = NotFound desc = could not find container \"b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9\": container with ID starting with b5777352b3ab0a2ae77cea9f80fc4612b23eb83399958c199154142749d9e0a9 not found: ID does not exist"
Jul 27 17:49:29.150702 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.150666    2563 scope.go:117] "RemoveContainer" containerID="e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76"
Jul 27 17:49:29.156006 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.155978    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"]
Jul 27 17:49:29.157783 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.157768    2563 scope.go:117] "RemoveContainer" containerID="0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0"
Jul 27 17:49:29.160877 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.160859    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78126-predictor-5c95549787-2v6h5"]
Jul 27 17:49:29.164896 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.164881    2563 scope.go:117] "RemoveContainer" containerID="e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76"
Jul 27 17:49:29.165107 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:29.165092    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76\": container with ID starting with e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76 not found: ID does not exist" containerID="e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76"
Jul 27 17:49:29.165154 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.165113    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76"} err="failed to get container status \"e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76\": rpc error: code = NotFound desc = could not find container \"e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76\": container with ID starting with e9ae4ad4f142edc3578f42ae77d164a7d6a7327a33f6b6ab1d9c315d4c091d76 not found: ID does not exist"
Jul 27 17:49:29.165154 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.165127    2563 scope.go:117] "RemoveContainer" containerID="0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0"
Jul 27 17:49:29.165392 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:49:29.165377    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0\": container with ID starting with 0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0 not found: ID does not exist" containerID="0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0"
Jul 27 17:49:29.165451 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.165395    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0"} err="failed to get container status \"0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0\": rpc error: code = NotFound desc = could not find container \"0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0\": container with ID starting with 0e3c91c4590b681ac59a7a665a0a926fa06baae1199f8328ed44b7bde97e99a0 not found: ID does not exist"
Jul 27 17:49:29.172959 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.172940    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"]
Jul 27 17:49:29.177644 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:29.177627    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78126-predictor-7fbf689c8b-ctxpl"]
Jul 27 17:49:30.796782 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:30.796748    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" path="/var/lib/kubelet/pods/5ff180e4-f789-4010-8388-3974ef9f061c/volumes"
Jul 27 17:49:30.797161 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:30.797147    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" path="/var/lib/kubelet/pods/fa91a475-0b92-48f1-8c4d-ec181f9fcac0/volumes"
Jul 27 17:49:33.130099 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:33.130073    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:49:33.130542 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:33.130522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:49:33.130614 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:33.130584    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 27 17:49:33.130842 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:33.130819    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:49:43.131161 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:43.131123    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 27 17:49:43.131599 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:43.131123    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:49:53.130608 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:53.130520    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 27 17:49:53.130952 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:49:53.130934    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:50:03.130996 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:03.130957    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 27 17:50:03.130996 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:03.130983    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:50:13.131988 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:13.131952    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:50:13.132422 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:13.132051    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:50:25.616105 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.616069    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"]
Jul 27 17:50:25.616557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.616359    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" containerID="cri-o://90ffbc56bb71671e74a5ce557aa4179cce4d16a5ba09f6e7e15f424707f8d914" gracePeriod=30
Jul 27 17:50:25.616557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.616428    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kube-rbac-proxy" containerID="cri-o://65ea9bf824df8a8a4ef8801b35ba916cf2feb8ff17387b9b5558486f7cb7ddff" gracePeriod=30
Jul 27 17:50:25.707463 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.707434    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"]
Jul 27 17:50:25.707690 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.707651    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" containerID="cri-o://0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674" gracePeriod=30
Jul 27 17:50:25.707766 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.707666    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kube-rbac-proxy" containerID="cri-o://8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c" gracePeriod=30
Jul 27 17:50:25.709145 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709122    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"]
Jul 27 17:50:25.709557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709539    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kube-rbac-proxy"
Jul 27 17:50:25.709557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709556    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kube-rbac-proxy"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709567    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kube-rbac-proxy"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709573    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kube-rbac-proxy"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709578    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709583    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709599    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709604    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709652    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kserve-container"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709661    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kserve-container"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709671    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff180e4-f789-4010-8388-3974ef9f061c" containerName="kube-rbac-proxy"
Jul 27 17:50:25.709696 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.709685    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa91a475-0b92-48f1-8c4d-ec181f9fcac0" containerName="kube-rbac-proxy"
Jul 27 17:50:25.714127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.714109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.720122 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.720101    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-60da7-kube-rbac-proxy-sar-config\""
Jul 27 17:50:25.720221 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.720141    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-60da7-predictor-serving-cert\""
Jul 27 17:50:25.726560 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.726536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"]
Jul 27 17:50:25.848953 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.848929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/746aab93-02eb-4983-8097-4add78798d59-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.849055 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.848973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/746aab93-02eb-4983-8097-4add78798d59-proxy-tls\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.849055 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.849014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4zd\" (UniqueName: \"kubernetes.io/projected/746aab93-02eb-4983-8097-4add78798d59-kube-api-access-zr4zd\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.855540 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.855515    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"]
Jul 27 17:50:25.858800 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.858786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:25.860577 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.860558    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-60da7-kube-rbac-proxy-sar-config\""
Jul 27 17:50:25.860681 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.860613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-60da7-predictor-serving-cert\""
Jul 27 17:50:25.867910 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.867868    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"]
Jul 27 17:50:25.950040 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/746aab93-02eb-4983-8097-4add78798d59-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.950185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp4cv\" (UniqueName: \"kubernetes.io/projected/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-api-access-bp4cv\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:25.950185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/746aab93-02eb-4983-8097-4add78798d59-proxy-tls\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.950185 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4zd\" (UniqueName: \"kubernetes.io/projected/746aab93-02eb-4983-8097-4add78798d59-kube-api-access-zr4zd\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.950429 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-proxy-tls\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:25.950555 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:25.950936 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.950915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/746aab93-02eb-4983-8097-4add78798d59-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.952623 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.952601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/746aab93-02eb-4983-8097-4add78798d59-proxy-tls\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:25.957415 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:25.957398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4zd\" (UniqueName: \"kubernetes.io/projected/746aab93-02eb-4983-8097-4add78798d59-kube-api-access-zr4zd\") pod \"success-200-isvc-60da7-predictor-667556ddc9-fl45j\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") " pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:26.030145 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.030108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:26.051437 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.051407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp4cv\" (UniqueName: \"kubernetes.io/projected/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-api-access-bp4cv\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.051555 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.051523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-proxy-tls\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.051627 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.051581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.052405 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.052381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.053744 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.053726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-proxy-tls\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.058542 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.058520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp4cv\" (UniqueName: \"kubernetes.io/projected/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-api-access-bp4cv\") pod \"error-404-isvc-60da7-predictor-68f4c8d978-66kdn\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") " pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.154265 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.154237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"]
Jul 27 17:50:26.156577 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:50:26.156554    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746aab93_02eb_4983_8097_4add78798d59.slice/crio-d0a394a5b299b9972cc8aba6eaf518d4e09e902b991d24e4dc582a33df26c781 WatchSource:0}: Error finding container d0a394a5b299b9972cc8aba6eaf518d4e09e902b991d24e4dc582a33df26c781: Status 404 returned error can't find the container with id d0a394a5b299b9972cc8aba6eaf518d4e09e902b991d24e4dc582a33df26c781
Jul 27 17:50:26.168739 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.168721    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:26.299689 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.299665    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"]
Jul 27 17:50:26.301868 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:50:26.301839    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ceab3bb_54e6_42a9_a3a7_6480a1e19fa1.slice/crio-a0aee84a0cb9a7b7d297b7e394c871a57088df5a114481b4dcd322a9247ed1db WatchSource:0}: Error finding container a0aee84a0cb9a7b7d297b7e394c871a57088df5a114481b4dcd322a9247ed1db: Status 404 returned error can't find the container with id a0aee84a0cb9a7b7d297b7e394c871a57088df5a114481b4dcd322a9247ed1db
Jul 27 17:50:26.382457 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.382433    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerID="65ea9bf824df8a8a4ef8801b35ba916cf2feb8ff17387b9b5558486f7cb7ddff" exitCode=2
Jul 27 17:50:26.382557 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.382503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" event={"ID":"7d32c32b-f978-4bf7-8ccf-6379858b4821","Type":"ContainerDied","Data":"65ea9bf824df8a8a4ef8801b35ba916cf2feb8ff17387b9b5558486f7cb7ddff"}
Jul 27 17:50:26.384174 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.384147    2563 generic.go:356] "Generic (PLEG): container finished" podID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerID="8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c" exitCode=2
Jul 27 17:50:26.384262 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.384217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" event={"ID":"2141d288-3ec1-4fb9-811d-598a3066e8ea","Type":"ContainerDied","Data":"8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c"}
Jul 27 17:50:26.386043 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.386019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" event={"ID":"746aab93-02eb-4983-8097-4add78798d59","Type":"ContainerStarted","Data":"63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867"}
Jul 27 17:50:26.386144 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.386063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" event={"ID":"746aab93-02eb-4983-8097-4add78798d59","Type":"ContainerStarted","Data":"47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa"}
Jul 27 17:50:26.386144 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.386093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" event={"ID":"746aab93-02eb-4983-8097-4add78798d59","Type":"ContainerStarted","Data":"d0a394a5b299b9972cc8aba6eaf518d4e09e902b991d24e4dc582a33df26c781"}
Jul 27 17:50:26.386144 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.386126    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:26.386271 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.386157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:26.387654 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.387630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" event={"ID":"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1","Type":"ContainerStarted","Data":"5773bb09ec16f9776d22ca6bec0e825bdc2d1539020f98115ea164bff78b1e85"}
Jul 27 17:50:26.387738 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.387686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" event={"ID":"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1","Type":"ContainerStarted","Data":"a0aee84a0cb9a7b7d297b7e394c871a57088df5a114481b4dcd322a9247ed1db"}
Jul 27 17:50:26.388023 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:26.388001    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 27 17:50:27.398976 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.398936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" event={"ID":"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1","Type":"ContainerStarted","Data":"83af28be17b17ff633293aa30dcb970ac163b43e36c9d64a8da59cab5b98ab99"}
Jul 27 17:50:27.399425 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.399119    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:27.399425 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.399184    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:27.399425 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.399320    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 27 17:50:27.400482 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.400459    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 27 17:50:27.414599 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.414559    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podStartSLOduration=2.414548534 podStartE2EDuration="2.414548534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:27.413519895 +0000 UTC m=+1385.124255560" watchObservedRunningTime="2026-07-27 17:50:27.414548534 +0000 UTC m=+1385.125284256"
Jul 27 17:50:27.415096 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:27.415067    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podStartSLOduration=2.415059862 podStartE2EDuration="2.415059862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:26.402035927 +0000 UTC m=+1384.112771607" watchObservedRunningTime="2026-07-27 17:50:27.415059862 +0000 UTC m=+1385.125795528"
Jul 27 17:50:28.027146 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.027107    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.30:8643/healthz\": dial tcp 10.132.0.30:8643: connect: connection refused"
Jul 27 17:50:28.027305 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.027113    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Jul 27 17:50:28.031435 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.031411    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Jul 27 17:50:28.031602 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.031584    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 27 17:50:28.406578 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.406542    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerID="90ffbc56bb71671e74a5ce557aa4179cce4d16a5ba09f6e7e15f424707f8d914" exitCode=0
Jul 27 17:50:28.406880 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.406584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" event={"ID":"7d32c32b-f978-4bf7-8ccf-6379858b4821","Type":"ContainerDied","Data":"90ffbc56bb71671e74a5ce557aa4179cce4d16a5ba09f6e7e15f424707f8d914"}
Jul 27 17:50:28.406964 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.406940    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 27 17:50:28.464964 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.464938    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:50:28.542440 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.542418    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:50:28.571965 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.571943    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbdtm\" (UniqueName: \"kubernetes.io/projected/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-api-access-xbdtm\") pod \"7d32c32b-f978-4bf7-8ccf-6379858b4821\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") "
Jul 27 17:50:28.572100 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.571996    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-rbac-proxy-sar-config\") pod \"7d32c32b-f978-4bf7-8ccf-6379858b4821\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") "
Jul 27 17:50:28.572100 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.572019    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls\") pod \"7d32c32b-f978-4bf7-8ccf-6379858b4821\" (UID: \"7d32c32b-f978-4bf7-8ccf-6379858b4821\") "
Jul 27 17:50:28.572382 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.572330    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7d32c32b-f978-4bf7-8ccf-6379858b4821" (UID: "7d32c32b-f978-4bf7-8ccf-6379858b4821"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:28.574017 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.573991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7d32c32b-f978-4bf7-8ccf-6379858b4821" (UID: "7d32c32b-f978-4bf7-8ccf-6379858b4821"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:50:28.574100 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.574019    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-api-access-xbdtm" (OuterVolumeSpecName: "kube-api-access-xbdtm") pod "7d32c32b-f978-4bf7-8ccf-6379858b4821" (UID: "7d32c32b-f978-4bf7-8ccf-6379858b4821"). InnerVolumeSpecName "kube-api-access-xbdtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:50:28.673313 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673260    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-rbac-proxy-sar-config\") pod \"2141d288-3ec1-4fb9-811d-598a3066e8ea\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") "
Jul 27 17:50:28.673313 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xc8m\" (UniqueName: \"kubernetes.io/projected/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-api-access-5xc8m\") pod \"2141d288-3ec1-4fb9-811d-598a3066e8ea\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") "
Jul 27 17:50:28.673461 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2141d288-3ec1-4fb9-811d-598a3066e8ea-proxy-tls\") pod \"2141d288-3ec1-4fb9-811d-598a3066e8ea\" (UID: \"2141d288-3ec1-4fb9-811d-598a3066e8ea\") "
Jul 27 17:50:28.673538 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673525    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:28.673599 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673541    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d32c32b-f978-4bf7-8ccf-6379858b4821-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:28.673599 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673553    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbdtm\" (UniqueName: \"kubernetes.io/projected/7d32c32b-f978-4bf7-8ccf-6379858b4821-kube-api-access-xbdtm\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:28.673599 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.673561    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2141d288-3ec1-4fb9-811d-598a3066e8ea" (UID: "2141d288-3ec1-4fb9-811d-598a3066e8ea"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:28.675222 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.675200    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2141d288-3ec1-4fb9-811d-598a3066e8ea-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2141d288-3ec1-4fb9-811d-598a3066e8ea" (UID: "2141d288-3ec1-4fb9-811d-598a3066e8ea"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:50:28.675291 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.675204    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-api-access-5xc8m" (OuterVolumeSpecName: "kube-api-access-5xc8m") pod "2141d288-3ec1-4fb9-811d-598a3066e8ea" (UID: "2141d288-3ec1-4fb9-811d-598a3066e8ea"). InnerVolumeSpecName "kube-api-access-5xc8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:50:28.774689 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.774668    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xc8m\" (UniqueName: \"kubernetes.io/projected/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-api-access-5xc8m\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:28.774689 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.774689    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2141d288-3ec1-4fb9-811d-598a3066e8ea-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:28.774812 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:28.774699    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2141d288-3ec1-4fb9-811d-598a3066e8ea-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:29.415178 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.415144    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"
Jul 27 17:50:29.415625 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.415141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf" event={"ID":"7d32c32b-f978-4bf7-8ccf-6379858b4821","Type":"ContainerDied","Data":"8bb1b31f91ca1b75d99034cd8f9686fe16420aad054253e7b2531f9e1812160e"}
Jul 27 17:50:29.415625 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.415290    2563 scope.go:117] "RemoveContainer" containerID="65ea9bf824df8a8a4ef8801b35ba916cf2feb8ff17387b9b5558486f7cb7ddff"
Jul 27 17:50:29.416867 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.416833    2563 generic.go:356] "Generic (PLEG): container finished" podID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerID="0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674" exitCode=0
Jul 27 17:50:29.416972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.416887    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"
Jul 27 17:50:29.416972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.416888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" event={"ID":"2141d288-3ec1-4fb9-811d-598a3066e8ea","Type":"ContainerDied","Data":"0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674"}
Jul 27 17:50:29.416972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.416949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz" event={"ID":"2141d288-3ec1-4fb9-811d-598a3066e8ea","Type":"ContainerDied","Data":"029e8957808227e495758841f00a5ec0447bc55b42db6838f40a05ef40a6d270"}
Jul 27 17:50:29.423708 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.423690    2563 scope.go:117] "RemoveContainer" containerID="90ffbc56bb71671e74a5ce557aa4179cce4d16a5ba09f6e7e15f424707f8d914"
Jul 27 17:50:29.431314 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.431295    2563 scope.go:117] "RemoveContainer" containerID="8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c"
Jul 27 17:50:29.432022 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.432004    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"]
Jul 27 17:50:29.435763 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.435721    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-796c8-predictor-79d76b5df7-q46rf"]
Jul 27 17:50:29.438688 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.438673    2563 scope.go:117] "RemoveContainer" containerID="0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674"
Jul 27 17:50:29.445012 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.444993    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"]
Jul 27 17:50:29.446573 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.446554    2563 scope.go:117] "RemoveContainer" containerID="8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c"
Jul 27 17:50:29.446890 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:29.446867    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c\": container with ID starting with 8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c not found: ID does not exist" containerID="8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c"
Jul 27 17:50:29.446977 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.446899    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c"} err="failed to get container status \"8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c\": rpc error: code = NotFound desc = could not find container \"8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c\": container with ID starting with 8b50e653a3a7d9a95d895a1b598440766e6869392c74b76ef2474eb73ad5d92c not found: ID does not exist"
Jul 27 17:50:29.446977 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.446938    2563 scope.go:117] "RemoveContainer" containerID="0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674"
Jul 27 17:50:29.447247 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:29.447221    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674\": container with ID starting with 0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674 not found: ID does not exist" containerID="0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674"
Jul 27 17:50:29.447451 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.447253    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674"} err="failed to get container status \"0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674\": rpc error: code = NotFound desc = could not find container \"0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674\": container with ID starting with 0dda43da618da4df7879f8ae1a329bea6d78898a0e33f7abd6a9a931ab2d5674 not found: ID does not exist"
Jul 27 17:50:29.448967 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:29.448935    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-796c8-predictor-5fb7fb7d5b-5sfzz"]
Jul 27 17:50:30.797073 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:30.797037    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" path="/var/lib/kubelet/pods/2141d288-3ec1-4fb9-811d-598a3066e8ea/volumes"
Jul 27 17:50:30.797464 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:30.797448    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" path="/var/lib/kubelet/pods/7d32c32b-f978-4bf7-8ccf-6379858b4821/volumes"
Jul 27 17:50:32.403353 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:32.403316    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:50:32.403770 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:32.403745    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 27 17:50:33.411927 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:33.411895    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:50:33.412355 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:33.412316    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 27 17:50:35.813451 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.813419    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"]
Jul 27 17:50:35.813880 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.813639    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" containerID="cri-o://a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3" gracePeriod=30
Jul 27 17:50:35.813880 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.813667    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kube-rbac-proxy" containerID="cri-o://5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535" gracePeriod=30
Jul 27 17:50:35.874152 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.874117    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"]
Jul 27 17:50:35.874408 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.874369    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container" containerID="cri-o://3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e" gracePeriod=30
Jul 27 17:50:35.874486 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.874399    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kube-rbac-proxy" containerID="cri-o://37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8" gracePeriod=30
Jul 27 17:50:35.875645 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.875621    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"]
Jul 27 17:50:35.875966 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.875950    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container"
Jul 27 17:50:35.875966 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.875967    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.875989    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kube-rbac-proxy"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.875995    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kube-rbac-proxy"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876013    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kube-rbac-proxy"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876019    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kube-rbac-proxy"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876028    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876036    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876088    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kube-rbac-proxy"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876096    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d32c32b-f978-4bf7-8ccf-6379858b4821" containerName="kserve-container"
Jul 27 17:50:35.876101 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876102    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kube-rbac-proxy"
Jul 27 17:50:35.876551 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.876113    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2141d288-3ec1-4fb9-811d-598a3066e8ea" containerName="kserve-container"
Jul 27 17:50:35.881069 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.881052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:35.882704 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.882687    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-28de7-predictor-serving-cert\""
Jul 27 17:50:35.882901 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.882887    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-28de7-kube-rbac-proxy-sar-config\""
Jul 27 17:50:35.886202 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.886177    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"]
Jul 27 17:50:35.967301 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.967272    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"]
Jul 27 17:50:35.970714 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.970696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:35.972553 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.972532    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-28de7-kube-rbac-proxy-sar-config\""
Jul 27 17:50:35.972620 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.972554    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-28de7-predictor-serving-cert\""
Jul 27 17:50:35.979758 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:35.979738    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"]
Jul 27 17:50:36.031787 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.031763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50cc817c-418f-483a-82d3-6ea7d7c97d5f-proxy-tls\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.031875 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.031804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgv97\" (UniqueName: \"kubernetes.io/projected/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-api-access-qgv97\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.031875 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.031822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.031947 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.031897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b62c\" (UniqueName: \"kubernetes.io/projected/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-api-access-7b62c\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.031987 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.031951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.032021 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.031986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.133296 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50cc817c-418f-483a-82d3-6ea7d7c97d5f-proxy-tls\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.133296 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgv97\" (UniqueName: \"kubernetes.io/projected/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-api-access-qgv97\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.133484 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.133539 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b62c\" (UniqueName: \"kubernetes.io/projected/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-api-access-7b62c\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.133590 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.133648 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.133722 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:36.133704    2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-28de7-predictor-serving-cert: secret "error-404-isvc-28de7-predictor-serving-cert" not found
Jul 27 17:50:36.133779 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:36.133764    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls podName:8ccb4422-a76d-4d68-9bd7-741ad5e33824 nodeName:}" failed. No retries permitted until 2026-07-27 17:50:36.633745567 +0000 UTC m=+1394.344481228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls") pod "error-404-isvc-28de7-predictor-6464688578-csjht" (UID: "8ccb4422-a76d-4d68-9bd7-741ad5e33824") : secret "error-404-isvc-28de7-predictor-serving-cert" not found
Jul 27 17:50:36.134017 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.133999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.134204 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.134186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.135586 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.135570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50cc817c-418f-483a-82d3-6ea7d7c97d5f-proxy-tls\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.140912 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.140893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b62c\" (UniqueName: \"kubernetes.io/projected/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-api-access-7b62c\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.141291 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.141269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgv97\" (UniqueName: \"kubernetes.io/projected/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-api-access-qgv97\") pod \"success-200-isvc-28de7-predictor-c4b44754c-xlf9g\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") " pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.196897 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.196871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.323769 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.323737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"]
Jul 27 17:50:36.326718 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:50:36.326691    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cc817c_418f_483a_82d3_6ea7d7c97d5f.slice/crio-5f3657e2420c3df9a1d95c6306d568d414d8d83b919710dbaeedfc5d7202f810 WatchSource:0}: Error finding container 5f3657e2420c3df9a1d95c6306d568d414d8d83b919710dbaeedfc5d7202f810: Status 404 returned error can't find the container with id 5f3657e2420c3df9a1d95c6306d568d414d8d83b919710dbaeedfc5d7202f810
Jul 27 17:50:36.453635 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.453598    2563 generic.go:356] "Generic (PLEG): container finished" podID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerID="5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535" exitCode=2
Jul 27 17:50:36.453802 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.453658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" event={"ID":"af339b57-0fcc-45f6-9452-7d294ba8ced2","Type":"ContainerDied","Data":"5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535"}
Jul 27 17:50:36.455637 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.455617    2563 generic.go:356] "Generic (PLEG): container finished" podID="592f360c-7d03-4896-90eb-abb9af963334" containerID="37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8" exitCode=2
Jul 27 17:50:36.455738 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.455680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" event={"ID":"592f360c-7d03-4896-90eb-abb9af963334","Type":"ContainerDied","Data":"37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8"}
Jul 27 17:50:36.457497 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.457472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" event={"ID":"50cc817c-418f-483a-82d3-6ea7d7c97d5f","Type":"ContainerStarted","Data":"5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940"}
Jul 27 17:50:36.457576 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.457525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" event={"ID":"50cc817c-418f-483a-82d3-6ea7d7c97d5f","Type":"ContainerStarted","Data":"534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62"}
Jul 27 17:50:36.457576 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.457556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" event={"ID":"50cc817c-418f-483a-82d3-6ea7d7c97d5f","Type":"ContainerStarted","Data":"5f3657e2420c3df9a1d95c6306d568d414d8d83b919710dbaeedfc5d7202f810"}
Jul 27 17:50:36.457653 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.457638    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:36.474564 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.474528    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podStartSLOduration=1.4745132650000001 podStartE2EDuration="1.474513265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:36.473284775 +0000 UTC m=+1394.184020444" watchObservedRunningTime="2026-07-27 17:50:36.474513265 +0000 UTC m=+1394.185248932"
Jul 27 17:50:36.637805 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.637738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.639923 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.639907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls\") pod \"error-404-isvc-28de7-predictor-6464688578-csjht\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") " pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:36.881252 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:36.881222    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:37.010694 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.010668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"]
Jul 27 17:50:37.013071 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:50:37.013042    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ccb4422_a76d_4d68_9bd7_741ad5e33824.slice/crio-80c66d71f0cd827766a5f8df63e4af7bae4a15ccfbc0d0a509ac157539b829f2 WatchSource:0}: Error finding container 80c66d71f0cd827766a5f8df63e4af7bae4a15ccfbc0d0a509ac157539b829f2: Status 404 returned error can't find the container with id 80c66d71f0cd827766a5f8df63e4af7bae4a15ccfbc0d0a509ac157539b829f2
Jul 27 17:50:37.463636 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.463604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" event={"ID":"8ccb4422-a76d-4d68-9bd7-741ad5e33824","Type":"ContainerStarted","Data":"fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842"}
Jul 27 17:50:37.463809 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.463674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" event={"ID":"8ccb4422-a76d-4d68-9bd7-741ad5e33824","Type":"ContainerStarted","Data":"ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2"}
Jul 27 17:50:37.463939 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.463921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:37.464026 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.463975    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:37.464026 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.464001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" event={"ID":"8ccb4422-a76d-4d68-9bd7-741ad5e33824","Type":"ContainerStarted","Data":"80c66d71f0cd827766a5f8df63e4af7bae4a15ccfbc0d0a509ac157539b829f2"}
Jul 27 17:50:37.464026 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.464027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:37.465045 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.465020    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 27 17:50:37.465124 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.465078    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 27 17:50:37.479464 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:37.479423    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podStartSLOduration=2.479410927 podStartE2EDuration="2.479410927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:37.479020563 +0000 UTC m=+1395.189756228" watchObservedRunningTime="2026-07-27 17:50:37.479410927 +0000 UTC m=+1395.190146592"
Jul 27 17:50:38.126327 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:38.126282    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 27 17:50:38.126761 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:38.126281    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.33:8643/healthz\": dial tcp 10.132.0.33:8643: connect: connection refused"
Jul 27 17:50:38.468158 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:38.468065    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 27 17:50:38.468158 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:38.468128    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 27 17:50:39.116871 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.116845    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:50:39.259048 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.258971    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls\") pod \"592f360c-7d03-4896-90eb-abb9af963334\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") "
Jul 27 17:50:39.259048 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.259008    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/592f360c-7d03-4896-90eb-abb9af963334-kube-rbac-proxy-sar-config\") pod \"592f360c-7d03-4896-90eb-abb9af963334\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") "
Jul 27 17:50:39.259048 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.259037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sc5bb\" (UniqueName: \"kubernetes.io/projected/592f360c-7d03-4896-90eb-abb9af963334-kube-api-access-sc5bb\") pod \"592f360c-7d03-4896-90eb-abb9af963334\" (UID: \"592f360c-7d03-4896-90eb-abb9af963334\") "
Jul 27 17:50:39.259567 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.259451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/592f360c-7d03-4896-90eb-abb9af963334-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "592f360c-7d03-4896-90eb-abb9af963334" (UID: "592f360c-7d03-4896-90eb-abb9af963334"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:39.261072 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.261049    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "592f360c-7d03-4896-90eb-abb9af963334" (UID: "592f360c-7d03-4896-90eb-abb9af963334"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:50:39.261153 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.261098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/592f360c-7d03-4896-90eb-abb9af963334-kube-api-access-sc5bb" (OuterVolumeSpecName: "kube-api-access-sc5bb") pod "592f360c-7d03-4896-90eb-abb9af963334" (UID: "592f360c-7d03-4896-90eb-abb9af963334"). InnerVolumeSpecName "kube-api-access-sc5bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:50:39.359646 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.359614    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/592f360c-7d03-4896-90eb-abb9af963334-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:39.359646 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.359644    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/592f360c-7d03-4896-90eb-abb9af963334-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:39.359831 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.359658    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc5bb\" (UniqueName: \"kubernetes.io/projected/592f360c-7d03-4896-90eb-abb9af963334-kube-api-access-sc5bb\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:39.473951 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.473921    2563 generic.go:356] "Generic (PLEG): container finished" podID="592f360c-7d03-4896-90eb-abb9af963334" containerID="3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e" exitCode=0
Jul 27 17:50:39.474129 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.473975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" event={"ID":"592f360c-7d03-4896-90eb-abb9af963334","Type":"ContainerDied","Data":"3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e"}
Jul 27 17:50:39.474129 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.473983    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"
Jul 27 17:50:39.474129 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.474020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g" event={"ID":"592f360c-7d03-4896-90eb-abb9af963334","Type":"ContainerDied","Data":"9bc272417e4cf2c6193cb24773838f99b20cdcac17da0fc87c0e621bf6121908"}
Jul 27 17:50:39.474129 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.474045    2563 scope.go:117] "RemoveContainer" containerID="37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8"
Jul 27 17:50:39.482935 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.482910    2563 scope.go:117] "RemoveContainer" containerID="3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e"
Jul 27 17:50:39.490857 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.490839    2563 scope.go:117] "RemoveContainer" containerID="37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8"
Jul 27 17:50:39.491095 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:39.491077    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8\": container with ID starting with 37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8 not found: ID does not exist" containerID="37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8"
Jul 27 17:50:39.491139 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.491103    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8"} err="failed to get container status \"37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8\": rpc error: code = NotFound desc = could not find container \"37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8\": container with ID starting with 37a42a4476086956f1079907bf7748f3c3ccb24c652fea65ea82eae33742f9b8 not found: ID does not exist"
Jul 27 17:50:39.491139 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.491120    2563 scope.go:117] "RemoveContainer" containerID="3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e"
Jul 27 17:50:39.491321 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:39.491305    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e\": container with ID starting with 3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e not found: ID does not exist" containerID="3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e"
Jul 27 17:50:39.491388 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.491324    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e"} err="failed to get container status \"3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e\": rpc error: code = NotFound desc = could not find container \"3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e\": container with ID starting with 3cb7cb30af215a266a9dafa98da6e00f354520f8b8a61b29a37f02dc136dc45e not found: ID does not exist"
Jul 27 17:50:39.495747 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.495727    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"]
Jul 27 17:50:39.500109 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:39.500061    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-37f6e-predictor-848b7c596-c679g"]
Jul 27 17:50:40.797178 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:40.797144    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="592f360c-7d03-4896-90eb-abb9af963334" path="/var/lib/kubelet/pods/592f360c-7d03-4896-90eb-abb9af963334/volumes"
Jul 27 17:50:42.404214 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:42.404178    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 27 17:50:43.126166 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.126125    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 27 17:50:43.131870 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.131845    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 27 17:50:43.260556 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.260533    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:50:43.392007 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.391919    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-rbac-proxy-sar-config\") pod \"af339b57-0fcc-45f6-9452-7d294ba8ced2\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") "
Jul 27 17:50:43.392007 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.391967    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-api-access-5wxll\") pod \"af339b57-0fcc-45f6-9452-7d294ba8ced2\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") "
Jul 27 17:50:43.392213 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.392023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls\") pod \"af339b57-0fcc-45f6-9452-7d294ba8ced2\" (UID: \"af339b57-0fcc-45f6-9452-7d294ba8ced2\") "
Jul 27 17:50:43.392289 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.392266    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "af339b57-0fcc-45f6-9452-7d294ba8ced2" (UID: "af339b57-0fcc-45f6-9452-7d294ba8ced2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:43.393928 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.393895    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "af339b57-0fcc-45f6-9452-7d294ba8ced2" (UID: "af339b57-0fcc-45f6-9452-7d294ba8ced2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:50:43.394030 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.393988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-api-access-5wxll" (OuterVolumeSpecName: "kube-api-access-5wxll") pod "af339b57-0fcc-45f6-9452-7d294ba8ced2" (UID: "af339b57-0fcc-45f6-9452-7d294ba8ced2"). InnerVolumeSpecName "kube-api-access-5wxll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:50:43.412698 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.412670    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 27 17:50:43.471735 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.471704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:50:43.472045 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.472028    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:50:43.472141 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.472115    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 27 17:50:43.472399 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.472376    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 27 17:50:43.492571 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.492547    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:43.492571 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.492571    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/af339b57-0fcc-45f6-9452-7d294ba8ced2-kube-api-access-5wxll\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:43.492706 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.492587    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af339b57-0fcc-45f6-9452-7d294ba8ced2-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:50:43.493840 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.493817    2563 generic.go:356] "Generic (PLEG): container finished" podID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerID="a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3" exitCode=0
Jul 27 17:50:43.493926 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.493853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" event={"ID":"af339b57-0fcc-45f6-9452-7d294ba8ced2","Type":"ContainerDied","Data":"a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3"}
Jul 27 17:50:43.493926 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.493875    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"
Jul 27 17:50:43.493926 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.493914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx" event={"ID":"af339b57-0fcc-45f6-9452-7d294ba8ced2","Type":"ContainerDied","Data":"fdcba921815f581998791b8069f88fffc8630e47c0b16f8351262a3a7c8a1194"}
Jul 27 17:50:43.494026 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.493954    2563 scope.go:117] "RemoveContainer" containerID="5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535"
Jul 27 17:50:43.502734 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.502716    2563 scope.go:117] "RemoveContainer" containerID="a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3"
Jul 27 17:50:43.509981 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.509965    2563 scope.go:117] "RemoveContainer" containerID="5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535"
Jul 27 17:50:43.510218 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:43.510204    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535\": container with ID starting with 5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535 not found: ID does not exist" containerID="5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535"
Jul 27 17:50:43.510266 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.510226    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535"} err="failed to get container status \"5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535\": rpc error: code = NotFound desc = could not find container \"5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535\": container with ID starting with 5c9aba3e95aa330c6957aeb4804bc2566528b374d7d36d7cc51013d574caf535 not found: ID does not exist"
Jul 27 17:50:43.510266 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.510242    2563 scope.go:117] "RemoveContainer" containerID="a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3"
Jul 27 17:50:43.510462 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:50:43.510444    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3\": container with ID starting with a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3 not found: ID does not exist" containerID="a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3"
Jul 27 17:50:43.510509 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.510466    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3"} err="failed to get container status \"a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3\": rpc error: code = NotFound desc = could not find container \"a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3\": container with ID starting with a57d9c7e4f651af12e576df757daab9d2aaff1f4f5173b6211b5858aa81ba7c3 not found: ID does not exist"
Jul 27 17:50:43.517739 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.517721    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"]
Jul 27 17:50:43.521252 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:43.521227    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-37f6e-predictor-54465b587c-v9rfx"]
Jul 27 17:50:44.796708 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:44.796677    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" path="/var/lib/kubelet/pods/af339b57-0fcc-45f6-9452-7d294ba8ced2/volumes"
Jul 27 17:50:52.404787 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:52.404746    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 27 17:50:53.412878 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:53.412838    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 27 17:50:53.472160 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:53.472128    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 27 17:50:53.472531 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:50:53.472506    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 27 17:51:02.404420 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:02.404379    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 27 17:51:03.413045 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:03.413005    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 27 17:51:03.472297 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:03.472263    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 27 17:51:03.472460 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:03.472405    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 27 17:51:12.405417 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:12.405385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:51:13.413646 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:13.413619    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:51:13.472202 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:13.472154    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 27 17:51:13.472494 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:13.472456    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 27 17:51:23.473527 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:23.473499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 17:51:23.473839 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:23.473587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 17:51:35.848116 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.848075    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"]
Jul 27 17:51:35.848546 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.848319    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container" containerID="cri-o://5773bb09ec16f9776d22ca6bec0e825bdc2d1539020f98115ea164bff78b1e85" gracePeriod=30
Jul 27 17:51:35.848546 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.848400    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kube-rbac-proxy" containerID="cri-o://83af28be17b17ff633293aa30dcb970ac163b43e36c9d64a8da59cab5b98ab99" gracePeriod=30
Jul 27 17:51:35.900664 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.900636    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"]
Jul 27 17:51:35.900910 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.900878    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container" containerID="cri-o://47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa" gracePeriod=30
Jul 27 17:51:35.901022 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.900893    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kube-rbac-proxy" containerID="cri-o://63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867" gracePeriod=30
Jul 27 17:51:35.970785 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.970760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"]
Jul 27 17:51:35.971085 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971074    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container"
Jul 27 17:51:35.971127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971087    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container"
Jul 27 17:51:35.971127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971102    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kube-rbac-proxy"
Jul 27 17:51:35.971127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971107    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kube-rbac-proxy"
Jul 27 17:51:35.971127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971115    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container"
Jul 27 17:51:35.971127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971120    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container"
Jul 27 17:51:35.971127 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971127    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kube-rbac-proxy"
Jul 27 17:51:35.971294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971132    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kube-rbac-proxy"
Jul 27 17:51:35.971294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971179    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kube-rbac-proxy"
Jul 27 17:51:35.971294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971187    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af339b57-0fcc-45f6-9452-7d294ba8ced2" containerName="kserve-container"
Jul 27 17:51:35.971294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971193    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kserve-container"
Jul 27 17:51:35.971294 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.971201    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="592f360c-7d03-4896-90eb-abb9af963334" containerName="kube-rbac-proxy"
Jul 27 17:51:35.973950 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.973935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:35.975739 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.975720    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-19a85-kube-rbac-proxy-sar-config\""
Jul 27 17:51:35.975917 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.975902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-19a85-predictor-serving-cert\""
Jul 27 17:51:35.986553 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:35.986405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"]
Jul 27 17:51:36.077774 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.077749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.077906 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.077779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxvpr\" (UniqueName: \"kubernetes.io/projected/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-api-access-sxvpr\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.077906 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.077817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/325f58ea-fbf5-46a1-8280-7df2accffdd6-proxy-tls\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.085204 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.085181    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"]
Jul 27 17:51:36.088465 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.088452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.090253 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.090234    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-19a85-predictor-serving-cert\""
Jul 27 17:51:36.090356 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.090236    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-19a85-kube-rbac-proxy-sar-config\""
Jul 27 17:51:36.096556 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.096533    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"]
Jul 27 17:51:36.179284 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0bf80531-e986-45d8-af30-104d9de00e0e-proxy-tls\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.179284 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvjps\" (UniqueName: \"kubernetes.io/projected/0bf80531-e986-45d8-af30-104d9de00e0e-kube-api-access-wvjps\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.179474 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0bf80531-e986-45d8-af30-104d9de00e0e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.179474 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.179474 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxvpr\" (UniqueName: \"kubernetes.io/projected/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-api-access-sxvpr\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.179601 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/325f58ea-fbf5-46a1-8280-7df2accffdd6-proxy-tls\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.179986 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.179963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.181804 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.181783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/325f58ea-fbf5-46a1-8280-7df2accffdd6-proxy-tls\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.187610 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.187590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxvpr\" (UniqueName: \"kubernetes.io/projected/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-api-access-sxvpr\") pod \"success-200-isvc-19a85-predictor-77b7868695-qcqp5\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") " pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.280092 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.280068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0bf80531-e986-45d8-af30-104d9de00e0e-proxy-tls\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.280183 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.280094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvjps\" (UniqueName: \"kubernetes.io/projected/0bf80531-e986-45d8-af30-104d9de00e0e-kube-api-access-wvjps\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.280183 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.280124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0bf80531-e986-45d8-af30-104d9de00e0e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.280743 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.280727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0bf80531-e986-45d8-af30-104d9de00e0e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.282145 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.282128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0bf80531-e986-45d8-af30-104d9de00e0e-proxy-tls\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.288972 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.288954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvjps\" (UniqueName: \"kubernetes.io/projected/0bf80531-e986-45d8-af30-104d9de00e0e-kube-api-access-wvjps\") pod \"error-404-isvc-19a85-predictor-57d9d7bf78-b79pr\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") " pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.289731 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.289710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.398840 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.398810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.415924 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.415900    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"]
Jul 27 17:51:36.418183 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:51:36.418159    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325f58ea_fbf5_46a1_8280_7df2accffdd6.slice/crio-9f1300ca4fdad355c304e9e0c3e64c5aac895b8c17bfe0a731aa7950ba0cd8c6 WatchSource:0}: Error finding container 9f1300ca4fdad355c304e9e0c3e64c5aac895b8c17bfe0a731aa7950ba0cd8c6: Status 404 returned error can't find the container with id 9f1300ca4fdad355c304e9e0c3e64c5aac895b8c17bfe0a731aa7950ba0cd8c6
Jul 27 17:51:36.556090 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.556061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"]
Jul 27 17:51:36.559139 ip-10-0-132-235 kubenswrapper[2563]: W0727 17:51:36.559116    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf80531_e986_45d8_af30_104d9de00e0e.slice/crio-767084cf7c0073f118399bbccd86b8b6cd613406d36e31a68ab661b817b52228 WatchSource:0}: Error finding container 767084cf7c0073f118399bbccd86b8b6cd613406d36e31a68ab661b817b52228: Status 404 returned error can't find the container with id 767084cf7c0073f118399bbccd86b8b6cd613406d36e31a68ab661b817b52228
Jul 27 17:51:36.738690 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.738625    2563 generic.go:356] "Generic (PLEG): container finished" podID="746aab93-02eb-4983-8097-4add78798d59" containerID="63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867" exitCode=2
Jul 27 17:51:36.738793 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.738706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" event={"ID":"746aab93-02eb-4983-8097-4add78798d59","Type":"ContainerDied","Data":"63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867"}
Jul 27 17:51:36.741038 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.740867    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerID="83af28be17b17ff633293aa30dcb970ac163b43e36c9d64a8da59cab5b98ab99" exitCode=2
Jul 27 17:51:36.741038 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.740974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" event={"ID":"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1","Type":"ContainerDied","Data":"83af28be17b17ff633293aa30dcb970ac163b43e36c9d64a8da59cab5b98ab99"}
Jul 27 17:51:36.745245 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.745220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" event={"ID":"325f58ea-fbf5-46a1-8280-7df2accffdd6","Type":"ContainerStarted","Data":"d52505c004aa17f29247ac9c7e1cb1fbe05e131c49ce68d8666f48923f4cd16c"}
Jul 27 17:51:36.745368 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.745277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" event={"ID":"325f58ea-fbf5-46a1-8280-7df2accffdd6","Type":"ContainerStarted","Data":"5daf80d6a60023e15a1bf1f75cc9725b01fca211eb7e84e3e888fc8a8372a834"}
Jul 27 17:51:36.745368 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.745313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" event={"ID":"325f58ea-fbf5-46a1-8280-7df2accffdd6","Type":"ContainerStarted","Data":"9f1300ca4fdad355c304e9e0c3e64c5aac895b8c17bfe0a731aa7950ba0cd8c6"}
Jul 27 17:51:36.746808 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.746247    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.746808 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.746296    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:36.748638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.748405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" event={"ID":"0bf80531-e986-45d8-af30-104d9de00e0e","Type":"ContainerStarted","Data":"58f71310c8623e97d0b341854b9dbf0ed8a5575497981959e807486b308115d7"}
Jul 27 17:51:36.748638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.748457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" event={"ID":"0bf80531-e986-45d8-af30-104d9de00e0e","Type":"ContainerStarted","Data":"149fed116e151b2a153a688677a946a7b5f0fa7d949e5e9bc875ccf5b175b615"}
Jul 27 17:51:36.748638 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.748490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" event={"ID":"0bf80531-e986-45d8-af30-104d9de00e0e","Type":"ContainerStarted","Data":"767084cf7c0073f118399bbccd86b8b6cd613406d36e31a68ab661b817b52228"}
Jul 27 17:51:36.749995 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.749002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.750417 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.750397    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:36.750723 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.750694    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:51:36.752569 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.752094    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:51:36.764463 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.764421    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podStartSLOduration=1.764409552 podStartE2EDuration="1.764409552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:51:36.762640425 +0000 UTC m=+1454.473376092" watchObservedRunningTime="2026-07-27 17:51:36.764409552 +0000 UTC m=+1454.475145217"
Jul 27 17:51:36.777681 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:36.777610    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podStartSLOduration=0.777594913 podStartE2EDuration="777.594913ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:51:36.777081334 +0000 UTC m=+1454.487816995" watchObservedRunningTime="2026-07-27 17:51:36.777594913 +0000 UTC m=+1454.488330581"
Jul 27 17:51:37.400030 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:37.399986    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Jul 27 17:51:37.754298 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:37.754213    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:51:37.754298 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:37.754216    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:51:38.407872 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.407835    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 27 17:51:38.760345 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.760305    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerID="5773bb09ec16f9776d22ca6bec0e825bdc2d1539020f98115ea164bff78b1e85" exitCode=0
Jul 27 17:51:38.760448 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.760368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" event={"ID":"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1","Type":"ContainerDied","Data":"5773bb09ec16f9776d22ca6bec0e825bdc2d1539020f98115ea164bff78b1e85"}
Jul 27 17:51:38.760894 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.760781    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:51:38.760894 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.760833    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:51:38.820074 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.820058    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:51:38.901854 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.901828    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp4cv\" (UniqueName: \"kubernetes.io/projected/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-api-access-bp4cv\") pod \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") "
Jul 27 17:51:38.901991 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.901914    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-proxy-tls\") pod \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") "
Jul 27 17:51:38.902055 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.902026    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-rbac-proxy-sar-config\") pod \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\" (UID: \"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1\") "
Jul 27 17:51:38.902329 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.902309    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" (UID: "9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:51:38.904353 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.904314    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" (UID: "9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:51:38.904353 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.904323    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-api-access-bp4cv" (OuterVolumeSpecName: "kube-api-access-bp4cv") pod "9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" (UID: "9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1"). InnerVolumeSpecName "kube-api-access-bp4cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:51:38.945288 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:38.945268    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:51:39.003062 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr4zd\" (UniqueName: \"kubernetes.io/projected/746aab93-02eb-4983-8097-4add78798d59-kube-api-access-zr4zd\") pod \"746aab93-02eb-4983-8097-4add78798d59\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") "
Jul 27 17:51:39.003177 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003137    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/746aab93-02eb-4983-8097-4add78798d59-proxy-tls\") pod \"746aab93-02eb-4983-8097-4add78798d59\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") "
Jul 27 17:51:39.003177 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/746aab93-02eb-4983-8097-4add78798d59-kube-rbac-proxy-sar-config\") pod \"746aab93-02eb-4983-8097-4add78798d59\" (UID: \"746aab93-02eb-4983-8097-4add78798d59\") "
Jul 27 17:51:39.003327 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003315    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:51:39.003424 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003331    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:51:39.003424 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003365    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp4cv\" (UniqueName: \"kubernetes.io/projected/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1-kube-api-access-bp4cv\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:51:39.003562 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.003542    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746aab93-02eb-4983-8097-4add78798d59-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "746aab93-02eb-4983-8097-4add78798d59" (UID: "746aab93-02eb-4983-8097-4add78798d59"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:51:39.004982 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.004954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746aab93-02eb-4983-8097-4add78798d59-kube-api-access-zr4zd" (OuterVolumeSpecName: "kube-api-access-zr4zd") pod "746aab93-02eb-4983-8097-4add78798d59" (UID: "746aab93-02eb-4983-8097-4add78798d59"). InnerVolumeSpecName "kube-api-access-zr4zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:51:39.005055 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.005001    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/746aab93-02eb-4983-8097-4add78798d59-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "746aab93-02eb-4983-8097-4add78798d59" (UID: "746aab93-02eb-4983-8097-4add78798d59"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:51:39.104677 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.104633    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr4zd\" (UniqueName: \"kubernetes.io/projected/746aab93-02eb-4983-8097-4add78798d59-kube-api-access-zr4zd\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:51:39.104677 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.104670    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/746aab93-02eb-4983-8097-4add78798d59-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:51:39.104677 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.104680    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/746aab93-02eb-4983-8097-4add78798d59-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 17:51:39.766305 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.766272    2563 generic.go:356] "Generic (PLEG): container finished" podID="746aab93-02eb-4983-8097-4add78798d59" containerID="47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa" exitCode=0
Jul 27 17:51:39.766669 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.766331    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"
Jul 27 17:51:39.766669 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.766369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" event={"ID":"746aab93-02eb-4983-8097-4add78798d59","Type":"ContainerDied","Data":"47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa"}
Jul 27 17:51:39.766669 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.766474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j" event={"ID":"746aab93-02eb-4983-8097-4add78798d59","Type":"ContainerDied","Data":"d0a394a5b299b9972cc8aba6eaf518d4e09e902b991d24e4dc582a33df26c781"}
Jul 27 17:51:39.766669 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.766518    2563 scope.go:117] "RemoveContainer" containerID="63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867"
Jul 27 17:51:39.768233 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.768210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn" event={"ID":"9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1","Type":"ContainerDied","Data":"a0aee84a0cb9a7b7d297b7e394c871a57088df5a114481b4dcd322a9247ed1db"}
Jul 27 17:51:39.768346 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.768240    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"
Jul 27 17:51:39.776257 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.776120    2563 scope.go:117] "RemoveContainer" containerID="47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa"
Jul 27 17:51:39.783744 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.783729    2563 scope.go:117] "RemoveContainer" containerID="63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867"
Jul 27 17:51:39.783985 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:51:39.783966    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867\": container with ID starting with 63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867 not found: ID does not exist" containerID="63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867"
Jul 27 17:51:39.784031 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.783994    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867"} err="failed to get container status \"63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867\": rpc error: code = NotFound desc = could not find container \"63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867\": container with ID starting with 63004afb8fb4b9c73226c2cd525379554df36bdc0f2d57b657f028b12bd6a867 not found: ID does not exist"
Jul 27 17:51:39.784031 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.784010    2563 scope.go:117] "RemoveContainer" containerID="47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa"
Jul 27 17:51:39.784240 ip-10-0-132-235 kubenswrapper[2563]: E0727 17:51:39.784222    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa\": container with ID starting with 47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa not found: ID does not exist" containerID="47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa"
Jul 27 17:51:39.784526 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.784246    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa"} err="failed to get container status \"47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa\": rpc error: code = NotFound desc = could not find container \"47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa\": container with ID starting with 47aa9bc8ca9a1e76862274a04e71f3875b80356b7493c9d4d73c722a81d384aa not found: ID does not exist"
Jul 27 17:51:39.784526 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.784260    2563 scope.go:117] "RemoveContainer" containerID="83af28be17b17ff633293aa30dcb970ac163b43e36c9d64a8da59cab5b98ab99"
Jul 27 17:51:39.790827 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.790807    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"]
Jul 27 17:51:39.791202 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.791186    2563 scope.go:117] "RemoveContainer" containerID="5773bb09ec16f9776d22ca6bec0e825bdc2d1539020f98115ea164bff78b1e85"
Jul 27 17:51:39.795425 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.795403    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-60da7-predictor-667556ddc9-fl45j"]
Jul 27 17:51:39.803590 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.803573    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"]
Jul 27 17:51:39.806028 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:39.806008    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-60da7-predictor-68f4c8d978-66kdn"]
Jul 27 17:51:40.796650 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:40.796622    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="746aab93-02eb-4983-8097-4add78798d59" path="/var/lib/kubelet/pods/746aab93-02eb-4983-8097-4add78798d59/volumes"
Jul 27 17:51:40.797035 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:40.797021    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" path="/var/lib/kubelet/pods/9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1/volumes"
Jul 27 17:51:43.765551 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:43.765524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:51:43.766136 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:43.766110    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:51:43.770547 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:43.770527    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:51:43.770941 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:43.770913    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:51:53.766027 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:53.765989    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:51:53.771269 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:51:53.771242    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:52:03.766109 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:03.766066    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:52:03.771372 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:03.771324    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:52:13.767519 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:13.766174    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 17:52:13.771244 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:13.771214    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 17:52:22.802559 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:22.802529    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:52:22.803986 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:22.803966    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:52:23.766509 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:23.766479    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 17:52:23.772412 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:52:23.772394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 17:57:22.831005 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:57:22.830968    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 17:57:22.831521 ip-10-0-132-235 kubenswrapper[2563]: I0727 17:57:22.831093    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:00:23.768251 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.768177    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"]
Jul 27 18:00:23.768797 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.768407    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container" containerID="cri-o://ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2" gracePeriod=30
Jul 27 18:00:23.768797 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.768443    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kube-rbac-proxy" containerID="cri-o://fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842" gracePeriod=30
Jul 27 18:00:23.810767 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.810740    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"]
Jul 27 18:00:23.811037 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.811009    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container" containerID="cri-o://534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62" gracePeriod=30
Jul 27 18:00:23.811240 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.811032    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kube-rbac-proxy" containerID="cri-o://5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940" gracePeriod=30
Jul 27 18:00:23.957349 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.957315    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerID="fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842" exitCode=2
Jul 27 18:00:23.957438 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.957385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" event={"ID":"8ccb4422-a76d-4d68-9bd7-741ad5e33824","Type":"ContainerDied","Data":"fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842"}
Jul 27 18:00:23.959032 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.959013    2563 generic.go:356] "Generic (PLEG): container finished" podID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerID="5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940" exitCode=2
Jul 27 18:00:23.959105 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:23.959069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" event={"ID":"50cc817c-418f-483a-82d3-6ea7d7c97d5f","Type":"ContainerDied","Data":"5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940"}
Jul 27 18:00:26.257286 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.257264    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 18:00:26.373917 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.373856    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-rbac-proxy-sar-config\") pod \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") "
Jul 27 18:00:26.373917 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.373907    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50cc817c-418f-483a-82d3-6ea7d7c97d5f-proxy-tls\") pod \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") "
Jul 27 18:00:26.374060 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.373927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgv97\" (UniqueName: \"kubernetes.io/projected/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-api-access-qgv97\") pod \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\" (UID: \"50cc817c-418f-483a-82d3-6ea7d7c97d5f\") "
Jul 27 18:00:26.374186 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.374166    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "50cc817c-418f-483a-82d3-6ea7d7c97d5f" (UID: "50cc817c-418f-483a-82d3-6ea7d7c97d5f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:26.375838 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.375813    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50cc817c-418f-483a-82d3-6ea7d7c97d5f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50cc817c-418f-483a-82d3-6ea7d7c97d5f" (UID: "50cc817c-418f-483a-82d3-6ea7d7c97d5f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:00:26.375957 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.375923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-api-access-qgv97" (OuterVolumeSpecName: "kube-api-access-qgv97") pod "50cc817c-418f-483a-82d3-6ea7d7c97d5f" (UID: "50cc817c-418f-483a-82d3-6ea7d7c97d5f"). InnerVolumeSpecName "kube-api-access-qgv97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:26.475083 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.475062    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:26.475083 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.475083    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50cc817c-418f-483a-82d3-6ea7d7c97d5f-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:26.475246 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.475092    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgv97\" (UniqueName: \"kubernetes.io/projected/50cc817c-418f-483a-82d3-6ea7d7c97d5f-kube-api-access-qgv97\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:26.973507 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.973479    2563 generic.go:356] "Generic (PLEG): container finished" podID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerID="534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62" exitCode=0
Jul 27 18:00:26.973632 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.973525    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"
Jul 27 18:00:26.973632 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.973527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" event={"ID":"50cc817c-418f-483a-82d3-6ea7d7c97d5f","Type":"ContainerDied","Data":"534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62"}
Jul 27 18:00:26.973724 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.973635    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g" event={"ID":"50cc817c-418f-483a-82d3-6ea7d7c97d5f","Type":"ContainerDied","Data":"5f3657e2420c3df9a1d95c6306d568d414d8d83b919710dbaeedfc5d7202f810"}
Jul 27 18:00:26.973724 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.973660    2563 scope.go:117] "RemoveContainer" containerID="5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940"
Jul 27 18:00:26.981669 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.981650    2563 scope.go:117] "RemoveContainer" containerID="534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62"
Jul 27 18:00:26.989518 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.989488    2563 scope.go:117] "RemoveContainer" containerID="5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940"
Jul 27 18:00:26.991546 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.990923    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"]
Jul 27 18:00:26.994082 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.994056    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-28de7-predictor-c4b44754c-xlf9g"]
Jul 27 18:00:26.994641 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:00:26.994613    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940\": container with ID starting with 5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940 not found: ID does not exist" containerID="5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940"
Jul 27 18:00:26.994722 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.994650    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940"} err="failed to get container status \"5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940\": rpc error: code = NotFound desc = could not find container \"5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940\": container with ID starting with 5365b4c2824e48b968ebfbb8f86464743b0b81a4271ccc6445110a98fad39940 not found: ID does not exist"
Jul 27 18:00:26.994800 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.994685    2563 scope.go:117] "RemoveContainer" containerID="534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62"
Jul 27 18:00:26.995044 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:00:26.995023    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62\": container with ID starting with 534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62 not found: ID does not exist" containerID="534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62"
Jul 27 18:00:26.995115 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:26.995049    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62"} err="failed to get container status \"534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62\": rpc error: code = NotFound desc = could not find container \"534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62\": container with ID starting with 534f038314f4d10769fd8238984c0ddfe46acb0dafe140bdb3777313e69d9a62 not found: ID does not exist"
Jul 27 18:00:28.468672 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:28.468635    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 27 18:00:28.796662 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:28.796629    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" path="/var/lib/kubelet/pods/50cc817c-418f-483a-82d3-6ea7d7c97d5f/volumes"
Jul 27 18:00:29.817589 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.817565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 18:00:29.897950 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.897927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b62c\" (UniqueName: \"kubernetes.io/projected/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-api-access-7b62c\") pod \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") "
Jul 27 18:00:29.898059 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.897974    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls\") pod \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") "
Jul 27 18:00:29.898116 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.898058    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-rbac-proxy-sar-config\") pod \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\" (UID: \"8ccb4422-a76d-4d68-9bd7-741ad5e33824\") "
Jul 27 18:00:29.898465 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.898441    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8ccb4422-a76d-4d68-9bd7-741ad5e33824" (UID: "8ccb4422-a76d-4d68-9bd7-741ad5e33824"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:29.899899 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.899877    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-api-access-7b62c" (OuterVolumeSpecName: "kube-api-access-7b62c") pod "8ccb4422-a76d-4d68-9bd7-741ad5e33824" (UID: "8ccb4422-a76d-4d68-9bd7-741ad5e33824"). InnerVolumeSpecName "kube-api-access-7b62c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:29.899992 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.899970    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8ccb4422-a76d-4d68-9bd7-741ad5e33824" (UID: "8ccb4422-a76d-4d68-9bd7-741ad5e33824"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:00:29.991385 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.991352    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerID="ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2" exitCode=0
Jul 27 18:00:29.991503 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.991394    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"
Jul 27 18:00:29.991503 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.991429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" event={"ID":"8ccb4422-a76d-4d68-9bd7-741ad5e33824","Type":"ContainerDied","Data":"ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2"}
Jul 27 18:00:29.991503 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.991484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht" event={"ID":"8ccb4422-a76d-4d68-9bd7-741ad5e33824","Type":"ContainerDied","Data":"80c66d71f0cd827766a5f8df63e4af7bae4a15ccfbc0d0a509ac157539b829f2"}
Jul 27 18:00:29.991628 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.991511    2563 scope.go:117] "RemoveContainer" containerID="fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842"
Jul 27 18:00:29.999525 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.999279    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:29.999525 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.999301    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b62c\" (UniqueName: \"kubernetes.io/projected/8ccb4422-a76d-4d68-9bd7-741ad5e33824-kube-api-access-7b62c\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:29.999525 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.999317    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ccb4422-a76d-4d68-9bd7-741ad5e33824-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:29.999525 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:29.999430    2563 scope.go:117] "RemoveContainer" containerID="ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2"
Jul 27 18:00:30.006748 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.006730    2563 scope.go:117] "RemoveContainer" containerID="fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842"
Jul 27 18:00:30.006993 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:00:30.006975    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842\": container with ID starting with fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842 not found: ID does not exist" containerID="fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842"
Jul 27 18:00:30.007046 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.007001    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842"} err="failed to get container status \"fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842\": rpc error: code = NotFound desc = could not find container \"fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842\": container with ID starting with fb5e1dd2ea837a9fa48bb3f56f6d6059c76f79a81fd2b70a0b07538f2383e842 not found: ID does not exist"
Jul 27 18:00:30.007046 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.007016    2563 scope.go:117] "RemoveContainer" containerID="ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2"
Jul 27 18:00:30.007263 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:00:30.007245    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2\": container with ID starting with ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2 not found: ID does not exist" containerID="ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2"
Jul 27 18:00:30.007304 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.007269    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2"} err="failed to get container status \"ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2\": rpc error: code = NotFound desc = could not find container \"ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2\": container with ID starting with ed1aa636537970ce6131d17a56f8109deb29d82aecf884108051851428ff87a2 not found: ID does not exist"
Jul 27 18:00:30.011538 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.011507    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"]
Jul 27 18:00:30.017497 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.017479    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-28de7-predictor-6464688578-csjht"]
Jul 27 18:00:30.796784 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:30.796753    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" path="/var/lib/kubelet/pods/8ccb4422-a76d-4d68-9bd7-741ad5e33824/volumes"
Jul 27 18:00:50.717985 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.717947    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"]
Jul 27 18:00:50.718467 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.718260    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" containerID="cri-o://149fed116e151b2a153a688677a946a7b5f0fa7d949e5e9bc875ccf5b175b615" gracePeriod=30
Jul 27 18:00:50.718467 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.718267    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kube-rbac-proxy" containerID="cri-o://58f71310c8623e97d0b341854b9dbf0ed8a5575497981959e807486b308115d7" gracePeriod=30
Jul 27 18:00:50.765941 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.765909    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"]
Jul 27 18:00:50.766160 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.766134    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" containerID="cri-o://5daf80d6a60023e15a1bf1f75cc9725b01fca211eb7e84e3e888fc8a8372a834" gracePeriod=30
Jul 27 18:00:50.766297 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.766259    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kube-rbac-proxy" containerID="cri-o://d52505c004aa17f29247ac9c7e1cb1fbe05e131c49ce68d8666f48923f4cd16c" gracePeriod=30
Jul 27 18:00:50.829125 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829093    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"]
Jul 27 18:00:50.829470 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829455    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829472    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829482    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829488    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829500    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829505    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829515    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829519 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829520    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829529    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829535    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829543    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829548    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829553    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829559    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829567    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829572    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829617    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829625    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829631    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829638    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kserve-container"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ceab3bb-54e6-42a9-a3a7-6480a1e19fa1" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829649    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cc817c-418f-483a-82d3-6ea7d7c97d5f" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829655    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="746aab93-02eb-4983-8097-4add78798d59" containerName="kube-rbac-proxy"
Jul 27 18:00:50.829728 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.829662    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ccb4422-a76d-4d68-9bd7-741ad5e33824" containerName="kube-rbac-proxy"
Jul 27 18:00:50.832599 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.832579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:50.834729 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.834705    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0a977-kube-rbac-proxy-sar-config\""
Jul 27 18:00:50.834850 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.834703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0a977-predictor-serving-cert\""
Jul 27 18:00:50.840955 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.840928    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"]
Jul 27 18:00:50.902950 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.902917    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"]
Jul 27 18:00:50.906248 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.906231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:50.908120 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.908094    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0a977-predictor-serving-cert\""
Jul 27 18:00:50.908227 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.908094    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0a977-kube-rbac-proxy-sar-config\""
Jul 27 18:00:50.914490 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.914469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"]
Jul 27 18:00:50.957832 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.957801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-proxy-tls\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:50.958028 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.957852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:50.958028 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:50.957946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsrq2\" (UniqueName: \"kubernetes.io/projected/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-api-access-fsrq2\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.058757 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.058723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsrq2\" (UniqueName: \"kubernetes.io/projected/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-api-access-fsrq2\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.058934 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.058772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.058934 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.058814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2ddj\" (UniqueName: \"kubernetes.io/projected/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-api-access-v2ddj\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.058934 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.058851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.059103 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.058935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-proxy-tls\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.059103 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.059013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.059655 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.059635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.061376 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.061354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-proxy-tls\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.066287 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.066262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsrq2\" (UniqueName: \"kubernetes.io/projected/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-api-access-fsrq2\") pod \"success-200-isvc-0a977-predictor-6db9658c4c-2z2cb\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") " pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.082541 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.082517    2563 generic.go:356] "Generic (PLEG): container finished" podID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerID="d52505c004aa17f29247ac9c7e1cb1fbe05e131c49ce68d8666f48923f4cd16c" exitCode=2
Jul 27 18:00:51.082670 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.082595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" event={"ID":"325f58ea-fbf5-46a1-8280-7df2accffdd6","Type":"ContainerDied","Data":"d52505c004aa17f29247ac9c7e1cb1fbe05e131c49ce68d8666f48923f4cd16c"}
Jul 27 18:00:51.084239 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.084218    2563 generic.go:356] "Generic (PLEG): container finished" podID="0bf80531-e986-45d8-af30-104d9de00e0e" containerID="58f71310c8623e97d0b341854b9dbf0ed8a5575497981959e807486b308115d7" exitCode=2
Jul 27 18:00:51.084378 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.084279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" event={"ID":"0bf80531-e986-45d8-af30-104d9de00e0e","Type":"ContainerDied","Data":"58f71310c8623e97d0b341854b9dbf0ed8a5575497981959e807486b308115d7"}
Jul 27 18:00:51.142770 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.142737    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:51.160140 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.160113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.160242 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.160158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2ddj\" (UniqueName: \"kubernetes.io/projected/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-api-access-v2ddj\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.160242 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.160183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.160375 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:00:51.160277    2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-0a977-predictor-serving-cert: secret "error-404-isvc-0a977-predictor-serving-cert" not found
Jul 27 18:00:51.160428 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:00:51.160376    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls podName:3ba2e80e-3913-49a9-af64-0577c451db0c nodeName:}" failed. No retries permitted until 2026-07-27 18:00:51.660354098 +0000 UTC m=+2009.371089758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls") pod "error-404-isvc-0a977-predictor-89f6898db-vhnk2" (UID: "3ba2e80e-3913-49a9-af64-0577c451db0c") : secret "error-404-isvc-0a977-predictor-serving-cert" not found
Jul 27 18:00:51.160775 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.160759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.168680 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.168660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2ddj\" (UniqueName: \"kubernetes.io/projected/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-api-access-v2ddj\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.277590 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.277562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"]
Jul 27 18:00:51.281469 ip-10-0-132-235 kubenswrapper[2563]: W0727 18:00:51.281445    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16fb8b9_c5c9_4737_9613_9c04c21fbd89.slice/crio-768dd71754bee6cb39d557e8fba29b907c06e6f876b570523f068251eaab6fc8 WatchSource:0}: Error finding container 768dd71754bee6cb39d557e8fba29b907c06e6f876b570523f068251eaab6fc8: Status 404 returned error can't find the container with id 768dd71754bee6cb39d557e8fba29b907c06e6f876b570523f068251eaab6fc8
Jul 27 18:00:51.664453 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.664369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.666720 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.666701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls\") pod \"error-404-isvc-0a977-predictor-89f6898db-vhnk2\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") " pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.817214 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.817170    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:51.946270 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:51.946244    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"]
Jul 27 18:00:51.949411 ip-10-0-132-235 kubenswrapper[2563]: W0727 18:00:51.949374    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba2e80e_3913_49a9_af64_0577c451db0c.slice/crio-b20ffb0dfca8cf9bd1bcbe25cf46d2c768367008affaf76b7db62b077683822d WatchSource:0}: Error finding container b20ffb0dfca8cf9bd1bcbe25cf46d2c768367008affaf76b7db62b077683822d: Status 404 returned error can't find the container with id b20ffb0dfca8cf9bd1bcbe25cf46d2c768367008affaf76b7db62b077683822d
Jul 27 18:00:52.090098 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.090064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" event={"ID":"3ba2e80e-3913-49a9-af64-0577c451db0c","Type":"ContainerStarted","Data":"e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c"}
Jul 27 18:00:52.090277 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.090128    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" event={"ID":"3ba2e80e-3913-49a9-af64-0577c451db0c","Type":"ContainerStarted","Data":"c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd"}
Jul 27 18:00:52.090277 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.090153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" event={"ID":"3ba2e80e-3913-49a9-af64-0577c451db0c","Type":"ContainerStarted","Data":"b20ffb0dfca8cf9bd1bcbe25cf46d2c768367008affaf76b7db62b077683822d"}
Jul 27 18:00:52.090277 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.090185    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:52.092663 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.092628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" event={"ID":"e16fb8b9-c5c9-4737-9613-9c04c21fbd89","Type":"ContainerStarted","Data":"f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f"}
Jul 27 18:00:52.092845 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.092744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" event={"ID":"e16fb8b9-c5c9-4737-9613-9c04c21fbd89","Type":"ContainerStarted","Data":"ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c"}
Jul 27 18:00:52.092845 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.092787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" event={"ID":"e16fb8b9-c5c9-4737-9613-9c04c21fbd89","Type":"ContainerStarted","Data":"768dd71754bee6cb39d557e8fba29b907c06e6f876b570523f068251eaab6fc8"}
Jul 27 18:00:52.095158 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.093773    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:52.095158 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.093838    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:52.095409 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.095198    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:00:52.106902 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.106853    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podStartSLOduration=2.106836718 podStartE2EDuration="2.106836718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:00:52.105542368 +0000 UTC m=+2009.816278037" watchObservedRunningTime="2026-07-27 18:00:52.106836718 +0000 UTC m=+2009.817572383"
Jul 27 18:00:52.121477 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:52.121437    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podStartSLOduration=2.121423851 podStartE2EDuration="2.121423851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:00:52.119798131 +0000 UTC m=+2009.830533798" watchObservedRunningTime="2026-07-27 18:00:52.121423851 +0000 UTC m=+2009.832159571"
Jul 27 18:00:53.097062 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.097024    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:53.097062 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.097052    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:00:53.098468 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.098442    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 27 18:00:53.761395 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.761356    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Jul 27 18:00:53.761578 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.761366    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Jul 27 18:00:53.766015 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.765987    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 27 18:00:53.770954 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:53.770932    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 27 18:00:54.103632 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.103599    2563 generic.go:356] "Generic (PLEG): container finished" podID="0bf80531-e986-45d8-af30-104d9de00e0e" containerID="149fed116e151b2a153a688677a946a7b5f0fa7d949e5e9bc875ccf5b175b615" exitCode=0
Jul 27 18:00:54.104031 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.103671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" event={"ID":"0bf80531-e986-45d8-af30-104d9de00e0e","Type":"ContainerDied","Data":"149fed116e151b2a153a688677a946a7b5f0fa7d949e5e9bc875ccf5b175b615"}
Jul 27 18:00:54.105703 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.105679    2563 generic.go:356] "Generic (PLEG): container finished" podID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerID="5daf80d6a60023e15a1bf1f75cc9725b01fca211eb7e84e3e888fc8a8372a834" exitCode=0
Jul 27 18:00:54.105832 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.105747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" event={"ID":"325f58ea-fbf5-46a1-8280-7df2accffdd6","Type":"ContainerDied","Data":"5daf80d6a60023e15a1bf1f75cc9725b01fca211eb7e84e3e888fc8a8372a834"}
Jul 27 18:00:54.106090 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.106067    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 27 18:00:54.106170 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.106070    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:00:54.183973 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.183949    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 18:00:54.186677 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.186656    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 18:00:54.285162 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxvpr\" (UniqueName: \"kubernetes.io/projected/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-api-access-sxvpr\") pod \"325f58ea-fbf5-46a1-8280-7df2accffdd6\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") "
Jul 27 18:00:54.285313 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285173    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/325f58ea-fbf5-46a1-8280-7df2accffdd6-proxy-tls\") pod \"325f58ea-fbf5-46a1-8280-7df2accffdd6\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") "
Jul 27 18:00:54.285313 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvjps\" (UniqueName: \"kubernetes.io/projected/0bf80531-e986-45d8-af30-104d9de00e0e-kube-api-access-wvjps\") pod \"0bf80531-e986-45d8-af30-104d9de00e0e\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") "
Jul 27 18:00:54.285313 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285230    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0bf80531-e986-45d8-af30-104d9de00e0e-kube-rbac-proxy-sar-config\") pod \"0bf80531-e986-45d8-af30-104d9de00e0e\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") "
Jul 27 18:00:54.285313 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285262    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-rbac-proxy-sar-config\") pod \"325f58ea-fbf5-46a1-8280-7df2accffdd6\" (UID: \"325f58ea-fbf5-46a1-8280-7df2accffdd6\") "
Jul 27 18:00:54.285313 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285299    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0bf80531-e986-45d8-af30-104d9de00e0e-proxy-tls\") pod \"0bf80531-e986-45d8-af30-104d9de00e0e\" (UID: \"0bf80531-e986-45d8-af30-104d9de00e0e\") "
Jul 27 18:00:54.285733 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285646    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bf80531-e986-45d8-af30-104d9de00e0e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0bf80531-e986-45d8-af30-104d9de00e0e" (UID: "0bf80531-e986-45d8-af30-104d9de00e0e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:54.285733 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.285712    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "325f58ea-fbf5-46a1-8280-7df2accffdd6" (UID: "325f58ea-fbf5-46a1-8280-7df2accffdd6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:54.287406 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.287362    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bf80531-e986-45d8-af30-104d9de00e0e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0bf80531-e986-45d8-af30-104d9de00e0e" (UID: "0bf80531-e986-45d8-af30-104d9de00e0e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:00:54.287513 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.287431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf80531-e986-45d8-af30-104d9de00e0e-kube-api-access-wvjps" (OuterVolumeSpecName: "kube-api-access-wvjps") pod "0bf80531-e986-45d8-af30-104d9de00e0e" (UID: "0bf80531-e986-45d8-af30-104d9de00e0e"). InnerVolumeSpecName "kube-api-access-wvjps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:54.287513 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.287434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/325f58ea-fbf5-46a1-8280-7df2accffdd6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "325f58ea-fbf5-46a1-8280-7df2accffdd6" (UID: "325f58ea-fbf5-46a1-8280-7df2accffdd6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:00:54.287780 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.287762    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-api-access-sxvpr" (OuterVolumeSpecName: "kube-api-access-sxvpr") pod "325f58ea-fbf5-46a1-8280-7df2accffdd6" (UID: "325f58ea-fbf5-46a1-8280-7df2accffdd6"). InnerVolumeSpecName "kube-api-access-sxvpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:54.385939 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.385896    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:54.385939 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.385929    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0bf80531-e986-45d8-af30-104d9de00e0e-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:54.385939 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.385945    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxvpr\" (UniqueName: \"kubernetes.io/projected/325f58ea-fbf5-46a1-8280-7df2accffdd6-kube-api-access-sxvpr\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:54.386182 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.385959    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/325f58ea-fbf5-46a1-8280-7df2accffdd6-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:54.386182 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.385971    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvjps\" (UniqueName: \"kubernetes.io/projected/0bf80531-e986-45d8-af30-104d9de00e0e-kube-api-access-wvjps\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:54.386182 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:54.385982    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0bf80531-e986-45d8-af30-104d9de00e0e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:00:55.111851 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.111768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"
Jul 27 18:00:55.112325 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.111771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5" event={"ID":"325f58ea-fbf5-46a1-8280-7df2accffdd6","Type":"ContainerDied","Data":"9f1300ca4fdad355c304e9e0c3e64c5aac895b8c17bfe0a731aa7950ba0cd8c6"}
Jul 27 18:00:55.112325 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.111911    2563 scope.go:117] "RemoveContainer" containerID="d52505c004aa17f29247ac9c7e1cb1fbe05e131c49ce68d8666f48923f4cd16c"
Jul 27 18:00:55.113620 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.113544    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"
Jul 27 18:00:55.113620 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.113549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr" event={"ID":"0bf80531-e986-45d8-af30-104d9de00e0e","Type":"ContainerDied","Data":"767084cf7c0073f118399bbccd86b8b6cd613406d36e31a68ab661b817b52228"}
Jul 27 18:00:55.120330 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.120308    2563 scope.go:117] "RemoveContainer" containerID="5daf80d6a60023e15a1bf1f75cc9725b01fca211eb7e84e3e888fc8a8372a834"
Jul 27 18:00:55.128178 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.128151    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"]
Jul 27 18:00:55.128545 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.128526    2563 scope.go:117] "RemoveContainer" containerID="58f71310c8623e97d0b341854b9dbf0ed8a5575497981959e807486b308115d7"
Jul 27 18:00:55.131985 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.131964    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-19a85-predictor-57d9d7bf78-b79pr"]
Jul 27 18:00:55.136682 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.136665    2563 scope.go:117] "RemoveContainer" containerID="149fed116e151b2a153a688677a946a7b5f0fa7d949e5e9bc875ccf5b175b615"
Jul 27 18:00:55.141284 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.141257    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"]
Jul 27 18:00:55.147363 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:55.147317    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-19a85-predictor-77b7868695-qcqp5"]
Jul 27 18:00:56.796882 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:56.796847    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" path="/var/lib/kubelet/pods/0bf80531-e986-45d8-af30-104d9de00e0e/volumes"
Jul 27 18:00:56.797285 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:56.797271    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" path="/var/lib/kubelet/pods/325f58ea-fbf5-46a1-8280-7df2accffdd6/volumes"
Jul 27 18:00:59.110504 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:59.110472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:00:59.110988 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:59.110962    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:00:59.111052 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:59.110989    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 27 18:00:59.111314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:00:59.111292    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:01:09.111688 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:09.111648    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:01:09.112114 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:09.111654    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 27 18:01:19.111257 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:19.111215    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 27 18:01:19.111739 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:19.111357    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:01:29.111193 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:29.111154    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 27 18:01:29.111785 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:29.111456    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 27 18:01:39.111485 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:39.111452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:01:39.111980 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:01:39.111958    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:02:22.862197 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:02:22.862111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:02:22.863173 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:02:22.863152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:07:22.886246 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:07:22.886219    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:07:22.888233 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:07:22.888210    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:12:22.910293 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:12:22.910210    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:12:22.912784 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:12:22.912766    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:17:22.934661 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:17:22.934554    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:17:22.938645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:17:22.938224    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:18:10.350366 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:10.350311    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"]
Jul 27 18:18:10.350822 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:10.350557    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container" containerID="cri-o://ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c" gracePeriod=30
Jul 27 18:18:10.350822 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:10.350616    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kube-rbac-proxy" containerID="cri-o://f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f" gracePeriod=30
Jul 27 18:18:10.401688 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:10.401649    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"]
Jul 27 18:18:10.401921 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:10.401888    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container" containerID="cri-o://c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd" gracePeriod=30
Jul 27 18:18:10.402325 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:10.401935    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kube-rbac-proxy" containerID="cri-o://e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c" gracePeriod=30
Jul 27 18:18:11.436842 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:11.436808    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerID="e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c" exitCode=2
Jul 27 18:18:11.437295 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:11.436885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" event={"ID":"3ba2e80e-3913-49a9-af64-0577c451db0c","Type":"ContainerDied","Data":"e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c"}
Jul 27 18:18:11.438674 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:11.438653    2563 generic.go:356] "Generic (PLEG): container finished" podID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerID="f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f" exitCode=2
Jul 27 18:18:11.438751 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:11.438685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" event={"ID":"e16fb8b9-c5c9-4737-9613-9c04c21fbd89","Type":"ContainerDied","Data":"f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f"}
Jul 27 18:18:13.327096 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.327066    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:18:13.444549 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.444523    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:18:13.448394 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.448371    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerID="c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd" exitCode=0
Jul 27 18:18:13.448491 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.448397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" event={"ID":"3ba2e80e-3913-49a9-af64-0577c451db0c","Type":"ContainerDied","Data":"c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd"}
Jul 27 18:18:13.448491 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.448415    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"
Jul 27 18:18:13.448491 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.448456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2" event={"ID":"3ba2e80e-3913-49a9-af64-0577c451db0c","Type":"ContainerDied","Data":"b20ffb0dfca8cf9bd1bcbe25cf46d2c768367008affaf76b7db62b077683822d"}
Jul 27 18:18:13.448491 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.448482    2563 scope.go:117] "RemoveContainer" containerID="e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c"
Jul 27 18:18:13.449961 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.449941    2563 generic.go:356] "Generic (PLEG): container finished" podID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerID="ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c" exitCode=0
Jul 27 18:18:13.450041 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.449971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" event={"ID":"e16fb8b9-c5c9-4737-9613-9c04c21fbd89","Type":"ContainerDied","Data":"ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c"}
Jul 27 18:18:13.450041 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.450002    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"
Jul 27 18:18:13.450041 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.450010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb" event={"ID":"e16fb8b9-c5c9-4737-9613-9c04c21fbd89","Type":"ContainerDied","Data":"768dd71754bee6cb39d557e8fba29b907c06e6f876b570523f068251eaab6fc8"}
Jul 27 18:18:13.456146 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.456129    2563 scope.go:117] "RemoveContainer" containerID="c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd"
Jul 27 18:18:13.464674 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.464651    2563 scope.go:117] "RemoveContainer" containerID="e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c"
Jul 27 18:18:13.465011 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:18:13.464982    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c\": container with ID starting with e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c not found: ID does not exist" containerID="e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c"
Jul 27 18:18:13.465100 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.465041    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c"} err="failed to get container status \"e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c\": rpc error: code = NotFound desc = could not find container \"e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c\": container with ID starting with e450435bd44f60b2493bfd1c7e3c119012e80f57f0e7aa2c215ea20e21e6d33c not found: ID does not exist"
Jul 27 18:18:13.465100 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.465065    2563 scope.go:117] "RemoveContainer" containerID="c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd"
Jul 27 18:18:13.465394 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:18:13.465374    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd\": container with ID starting with c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd not found: ID does not exist" containerID="c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd"
Jul 27 18:18:13.465458 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.465404    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd"} err="failed to get container status \"c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd\": rpc error: code = NotFound desc = could not find container \"c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd\": container with ID starting with c5bd6ff8dea79f74c85658cc18be29ae9aa50b3327a16344c563ac1de6fa4dcd not found: ID does not exist"
Jul 27 18:18:13.465458 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.465423    2563 scope.go:117] "RemoveContainer" containerID="f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f"
Jul 27 18:18:13.472632 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.472601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsrq2\" (UniqueName: \"kubernetes.io/projected/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-api-access-fsrq2\") pod \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") "
Jul 27 18:18:13.472704 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.472657    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-proxy-tls\") pod \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") "
Jul 27 18:18:13.472704 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.472672    2563 scope.go:117] "RemoveContainer" containerID="ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c"
Jul 27 18:18:13.472704 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.472692    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-rbac-proxy-sar-config\") pod \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\" (UID: \"e16fb8b9-c5c9-4737-9613-9c04c21fbd89\") "
Jul 27 18:18:13.473143 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.473116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e16fb8b9-c5c9-4737-9613-9c04c21fbd89" (UID: "e16fb8b9-c5c9-4737-9613-9c04c21fbd89"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:18:13.475152 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.475118    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e16fb8b9-c5c9-4737-9613-9c04c21fbd89" (UID: "e16fb8b9-c5c9-4737-9613-9c04c21fbd89"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:18:13.475152 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.475139    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-api-access-fsrq2" (OuterVolumeSpecName: "kube-api-access-fsrq2") pod "e16fb8b9-c5c9-4737-9613-9c04c21fbd89" (UID: "e16fb8b9-c5c9-4737-9613-9c04c21fbd89"). InnerVolumeSpecName "kube-api-access-fsrq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:18:13.482284 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.482263    2563 scope.go:117] "RemoveContainer" containerID="f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f"
Jul 27 18:18:13.482572 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:18:13.482544    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f\": container with ID starting with f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f not found: ID does not exist" containerID="f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f"
Jul 27 18:18:13.482651 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.482582    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f"} err="failed to get container status \"f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f\": rpc error: code = NotFound desc = could not find container \"f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f\": container with ID starting with f452c5e82acf5285daf7b42abd3dbb756926c86dc526af01af43d508b0dfc68f not found: ID does not exist"
Jul 27 18:18:13.482651 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.482604    2563 scope.go:117] "RemoveContainer" containerID="ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c"
Jul 27 18:18:13.482916 ip-10-0-132-235 kubenswrapper[2563]: E0727 18:18:13.482892    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c\": container with ID starting with ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c not found: ID does not exist" containerID="ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c"
Jul 27 18:18:13.482998 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.482924    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c"} err="failed to get container status \"ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c\": rpc error: code = NotFound desc = could not find container \"ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c\": container with ID starting with ae1f8233929f820770906ee30c913be4b82f2b4a15c027002cf5132ee15e440c not found: ID does not exist"
Jul 27 18:18:13.573237 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-rbac-proxy-sar-config\") pod \"3ba2e80e-3913-49a9-af64-0577c451db0c\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") "
Jul 27 18:18:13.573351 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573262    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2ddj\" (UniqueName: \"kubernetes.io/projected/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-api-access-v2ddj\") pod \"3ba2e80e-3913-49a9-af64-0577c451db0c\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") "
Jul 27 18:18:13.573351 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls\") pod \"3ba2e80e-3913-49a9-af64-0577c451db0c\" (UID: \"3ba2e80e-3913-49a9-af64-0577c451db0c\") "
Jul 27 18:18:13.573523 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573476    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsrq2\" (UniqueName: \"kubernetes.io/projected/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-api-access-fsrq2\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:18:13.573523 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573487    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:18:13.573523 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573496    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e16fb8b9-c5c9-4737-9613-9c04c21fbd89-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:18:13.573678 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.573641    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3ba2e80e-3913-49a9-af64-0577c451db0c" (UID: "3ba2e80e-3913-49a9-af64-0577c451db0c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:18:13.575235 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.575214    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-api-access-v2ddj" (OuterVolumeSpecName: "kube-api-access-v2ddj") pod "3ba2e80e-3913-49a9-af64-0577c451db0c" (UID: "3ba2e80e-3913-49a9-af64-0577c451db0c"). InnerVolumeSpecName "kube-api-access-v2ddj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:18:13.575235 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.575223    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3ba2e80e-3913-49a9-af64-0577c451db0c" (UID: "3ba2e80e-3913-49a9-af64-0577c451db0c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:18:13.673955 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.673931    2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:18:13.673955 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.673952    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2ddj\" (UniqueName: \"kubernetes.io/projected/3ba2e80e-3913-49a9-af64-0577c451db0c-kube-api-access-v2ddj\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:18:13.674069 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.673962    2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ba2e80e-3913-49a9-af64-0577c451db0c-proxy-tls\") on node \"ip-10-0-132-235.ec2.internal\" DevicePath \"\""
Jul 27 18:18:13.773205 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.773182    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"]
Jul 27 18:18:13.774970 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.774948    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a977-predictor-89f6898db-vhnk2"]
Jul 27 18:18:13.785644 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.785619    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"]
Jul 27 18:18:13.787427 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:13.787408    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0a977-predictor-6db9658c4c-2z2cb"]
Jul 27 18:18:14.797610 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:14.797577    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" path="/var/lib/kubelet/pods/3ba2e80e-3913-49a9-af64-0577c451db0c/volumes"
Jul 27 18:18:14.797981 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:14.797968    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" path="/var/lib/kubelet/pods/e16fb8b9-c5c9-4737-9613-9c04c21fbd89/volumes"
Jul 27 18:18:46.510951 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:46.510911    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-phrw4_04a2888d-ead2-4657-a191-b7fe4d26c689/global-pull-secret-syncer/0.log"
Jul 27 18:18:46.511527 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:46.511292    2563 ???:1] "http: TLS handshake error from 10.0.140.185:48790: EOF"
Jul 27 18:18:46.618709 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:46.618683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jxl2n_e19ebf9f-ccb5-400e-b6f0-d2f8a2fdb44e/konnectivity-agent/0.log"
Jul 27 18:18:46.719943 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:46.719916    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-235.ec2.internal_1eab52b5abcc336338b0570279cda8a5/haproxy/0.log"
Jul 27 18:18:51.132870 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:51.132840    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-xbnbm_80a87ec3-31ec-4ddb-ae24-b9ef51e9c227/cluster-monitoring-operator/0.log"
Jul 27 18:18:51.234723 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:51.234664    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6785ff5849-q65rt_f2d95b69-a34f-4a3d-80e7-d7d84527813b/metrics-server/0.log"
Jul 27 18:18:51.305455 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:51.305425    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f4k9p_ad14d593-83c0-4c43-8837-9aaf1fb10e8e/node-exporter/0.log"
Jul 27 18:18:51.324431 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:51.324408    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f4k9p_ad14d593-83c0-4c43-8837-9aaf1fb10e8e/kube-rbac-proxy/0.log"
Jul 27 18:18:51.346148 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:51.346128    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f4k9p_ad14d593-83c0-4c43-8837-9aaf1fb10e8e/init-textfile/0.log"
Jul 27 18:18:54.011308 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:54.011272    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/1.log"
Jul 27 18:18:54.022686 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:54.022636    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxvl2_f4417e95-0dab-43b2-aea7-e774b713f5fe/console-operator/2.log"
Jul 27 18:18:54.986877 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:54.986850    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-lflhc_9f754619-6367-4fd6-ab8a-4c9367f5bb6f/volume-data-source-validator/0.log"
Jul 27 18:18:55.076658 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.076626    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"]
Jul 27 18:18:55.077008 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.076954    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container"
Jul 27 18:18:55.077008 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.076973    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container"
Jul 27 18:18:55.077008 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.076987    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077008 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.076997    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077008 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077006    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077008 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077012    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077023    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077028    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077035    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077041    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077050    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077070    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077085    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077094    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077103    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077108    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077162    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077172    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077180    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077185    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba2e80e-3913-49a9-af64-0577c451db0c" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077192    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16fb8b9-c5c9-4737-9613-9c04c21fbd89" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077197    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kube-rbac-proxy"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077202    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="325f58ea-fbf5-46a1-8280-7df2accffdd6" containerName="kserve-container"
Jul 27 18:18:55.077314 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.077207    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bf80531-e986-45d8-af30-104d9de00e0e" containerName="kserve-container"
Jul 27 18:18:55.080260 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.080243    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.082291 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.082266    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mlzzr\"/\"openshift-service-ca.crt\""
Jul 27 18:18:55.082845 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.082822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mlzzr\"/\"default-dockercfg-hfqqn\""
Jul 27 18:18:55.082963 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.082850    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mlzzr\"/\"kube-root-ca.crt\""
Jul 27 18:18:55.087016 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.086949    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"]
Jul 27 18:18:55.153091 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.153070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-proc\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.153195 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.153097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-lib-modules\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.153195 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.153115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-podres\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.153195 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.153143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r5xw\" (UniqueName: \"kubernetes.io/projected/13383370-4c51-464a-94c5-16742970e4e4-kube-api-access-5r5xw\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.153195 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.153180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-sys\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254457 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-proc\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254457 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-lib-modules\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254457 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-podres\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r5xw\" (UniqueName: \"kubernetes.io/projected/13383370-4c51-464a-94c5-16742970e4e4-kube-api-access-5r5xw\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-sys\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-proc\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-sys\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-podres\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.254645 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.254582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13383370-4c51-464a-94c5-16742970e4e4-lib-modules\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.262352 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.262323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r5xw\" (UniqueName: \"kubernetes.io/projected/13383370-4c51-464a-94c5-16742970e4e4-kube-api-access-5r5xw\") pod \"perf-node-gather-daemonset-mx6nw\" (UID: \"13383370-4c51-464a-94c5-16742970e4e4\") " pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.390550 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.390529    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.516381 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.516353    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"]
Jul 27 18:18:55.519069 ip-10-0-132-235 kubenswrapper[2563]: W0727 18:18:55.519043    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod13383370_4c51_464a_94c5_16742970e4e4.slice/crio-6f0c7fac31264e00633748ba5071a0bf0f3c099691ade383f9d907f9d3139e18 WatchSource:0}: Error finding container 6f0c7fac31264e00633748ba5071a0bf0f3c099691ade383f9d907f9d3139e18: Status 404 returned error can't find the container with id 6f0c7fac31264e00633748ba5071a0bf0f3c099691ade383f9d907f9d3139e18
Jul 27 18:18:55.619432 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.619403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw" event={"ID":"13383370-4c51-464a-94c5-16742970e4e4","Type":"ContainerStarted","Data":"2de44f03cc0d85d92b2e9f13a5e77b31b816a7f451b3bbc5372bdd7ea7955223"}
Jul 27 18:18:55.619541 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.619454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw" event={"ID":"13383370-4c51-464a-94c5-16742970e4e4","Type":"ContainerStarted","Data":"6f0c7fac31264e00633748ba5071a0bf0f3c099691ade383f9d907f9d3139e18"}
Jul 27 18:18:55.619541 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.619520    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:18:55.633294 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.633253    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw" podStartSLOduration=0.633239685 podStartE2EDuration="633.239685ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:18:55.632647985 +0000 UTC m=+3093.343383651" watchObservedRunningTime="2026-07-27 18:18:55.633239685 +0000 UTC m=+3093.343975351"
Jul 27 18:18:55.977157 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.976958    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sk7vl_25394289-0555-486d-a7a4-bf37a57caf8a/dns/0.log"
Jul 27 18:18:55.996626 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:55.996604    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sk7vl_25394289-0555-486d-a7a4-bf37a57caf8a/kube-rbac-proxy/0.log"
Jul 27 18:18:56.114910 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:56.114884    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjcvx_78a9220d-c136-489e-84db-36da1c4423dc/dns-node-resolver/0.log"
Jul 27 18:18:56.645444 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:56.645387    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wk5wk_56a1b52a-257c-48a7-8a73-e59e27e1a1d3/node-ca/0.log"
Jul 27 18:18:57.386081 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:57.386051    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dbb545db7-fnmlq_0e810fcc-7e49-4772-9ade-c9731493e4f7/router/0.log"
Jul 27 18:18:57.716124 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:57.716014    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dkhl7_6d90298b-c573-4d45-accc-086b982b6d8e/serve-healthcheck-canary/0.log"
Jul 27 18:18:58.124027 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:58.123996    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-jwmmm_d696cb3d-caf7-4bfe-876d-eca689f294f0/insights-operator/0.log"
Jul 27 18:18:58.126364 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:58.126295    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-jwmmm_d696cb3d-caf7-4bfe-876d-eca689f294f0/insights-operator/1.log"
Jul 27 18:18:58.214006 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:58.213987    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qdqnz_a49911cf-a59c-449e-8fc5-14128db1c9e4/kube-rbac-proxy/0.log"
Jul 27 18:18:58.235593 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:58.235561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qdqnz_a49911cf-a59c-449e-8fc5-14128db1c9e4/exporter/0.log"
Jul 27 18:18:58.254871 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:18:58.254851    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qdqnz_a49911cf-a59c-449e-8fc5-14128db1c9e4/extractor/0.log"
Jul 27 18:19:01.633686 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:01.633658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mlzzr/perf-node-gather-daemonset-mx6nw"
Jul 27 18:19:05.227985 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:05.227909    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv_91c5285c-1b1f-4512-874a-5d4fc63cb925/kube-storage-version-migrator-operator/1.log"
Jul 27 18:19:05.229563 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:05.229531    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-h2pbv_91c5285c-1b1f-4512-874a-5d4fc63cb925/kube-storage-version-migrator-operator/0.log"
Jul 27 18:19:06.113122 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.113096    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/kube-multus-additional-cni-plugins/0.log"
Jul 27 18:19:06.136523 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.136498    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/egress-router-binary-copy/0.log"
Jul 27 18:19:06.155285 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.155264    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/cni-plugins/0.log"
Jul 27 18:19:06.173873 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.173820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/bond-cni-plugin/0.log"
Jul 27 18:19:06.192226 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.192204    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/routeoverride-cni/0.log"
Jul 27 18:19:06.211085 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.211065    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/whereabouts-cni-bincopy/0.log"
Jul 27 18:19:06.230272 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.230249    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hzgdr_4d69785c-df22-4d63-86f2-e60bfb5297f1/whereabouts-cni/0.log"
Jul 27 18:19:06.638259 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.638225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r82m9_141a7e0f-fdc5-4152-9626-c6387bece93a/kube-multus/0.log"
Jul 27 18:19:06.751244 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.751191    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n44f9_269531c4-3fdf-4ceb-8046-2228ac3c143d/network-metrics-daemon/0.log"
Jul 27 18:19:06.767026 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:06.767004    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n44f9_269531c4-3fdf-4ceb-8046-2228ac3c143d/kube-rbac-proxy/0.log"
Jul 27 18:19:07.464924 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.464885    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/ovn-controller/0.log"
Jul 27 18:19:07.483133 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.483107    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/ovn-acl-logging/0.log"
Jul 27 18:19:07.506429 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.506390    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/kube-rbac-proxy-node/0.log"
Jul 27 18:19:07.525672 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.525649    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/kube-rbac-proxy-ovn-metrics/0.log"
Jul 27 18:19:07.541999 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.541976    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/northd/0.log"
Jul 27 18:19:07.561696 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.561675    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/nbdb/0.log"
Jul 27 18:19:07.581499 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.581473    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/sbdb/0.log"
Jul 27 18:19:07.646594 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:07.646575    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxgdr_9066e682-b257-499f-8c53-b61cc205317b/ovnkube-controller/0.log"
Jul 27 18:19:09.034374 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:09.034322    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-bg78h_5a6f3ad6-72d9-473c-b421-1a4023ef8e9a/check-endpoints/0.log"
Jul 27 18:19:09.055954 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:09.055925    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-dxf4t_2e5f8925-e357-4ef5-9b0b-1644bfe959ff/network-check-target-container/0.log"
Jul 27 18:19:10.065602 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:10.065576    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-grtsm_8193031b-4bdb-4756-9607-4fe816a48535/iptables-alerter/0.log"
Jul 27 18:19:10.682387 ip-10-0-132-235 kubenswrapper[2563]: I0727 18:19:10.682359    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-b4wnd_1d4ac820-1268-4587-b1a8-fa9ad85ba912/tuned/0.log"